Minimum Entropy Research Articles

Colloidal particles as noise source for random number generation

In this work, we investigate colloidal particle patterns as a possible noise source for random number generation. We systematically analyze the minimum entropy of the noise source over different particle concentrations of {1, 3, 5, 7, 10, 12, 15} mg/ml according to the recommendations of the National Institute of Standards and Technology Special Publication 800-90B. The estimated minimum entropy of the non-independent and identically distributed particle pattern noise source is Hmin = 0.5896/1 bit at a particle amount of 5 mg/ml. For further entropy extraction on the noise source data, the secure hash algorithm is used to construct an entropy source. The randomness of the derived entropy source is verified according to the National Institute of Standards and Technology Special Publication 800-22 Rev. 1a and the accompanying statistical test suite. The entropy source passes all randomness tests of the statistical test suite and shows an estimated minimum entropy of Hmin = 0.9992/1 bit.

Dynamic graph attention-guided graph clustering with entropy minimization self-supervision

Dynamic graph attention-guided graph clustering with entropy minimization self-supervision

Transitions between equilibrium and nonequilibrium phenomena in the description of crystal growth

The close intertwining of equilibrium and nonequilibrium thermodynamic representations and transitions between the two limiting principles of thermodynamics: the second beginning and the principle of least coercion (minimum entropy production in the stationary regime) constitute the main content of phenomenological theories of crystal growth. The difference of basic postulates of two sections of thermodynamics forces to discuss problems of reversibility and irreversibility of time, scales of observed phenomena and rules of conjugation of thermodynamic forces and flows in theories of crystal growth. A variant of the solution of some conjugation problems is shown on the example of the fluctuation model of dislocation crystal growth, which is based on the stationary isothermal process of thermodynamic free energy fluctuations. In the case of the limiting mode of adsorption of impurities on the crystal face according to the Langmuir model, the free energy fluctuations possessing the absence of the memory effect allow us to identify three chemical potentials of building particles that determine the corresponding values of solution supersaturations realized at different scale levels at the growing crystal face containing a helical dislocation. The supersaturations control quasi-equilibrium and nonequilibrium thermodynamic processes that constitute a single dislocation mechanism of crystal growth.

LesionMix data enhancement and entropy minimization for semi-supervised lesion segmentation of lung cancer

Determining the location and contour of the lesion is a crucial prerequisite for medical diagnosis, subsequent personalized treatment plan and prognostic prediction of lung cancer. Semi-supervised learning and data augmentation methods facilitate deep learning to be used in many fields of medical imaging. In this paper, we introduce a novel data enhancement technique called LesionMix. This method involves extracting and reusing lesions from a limited amount of labeled CT data, thereby enhancing the efficiency of utilizing those labeled data. Meanwhile, we propose a two-stage semi-supervised training strategy called Entropy Minimization LesionMix (EMLM). In the first stage, features containing lesion contour information are rapidly learned through LesionMix data augmentation. Entropy minimization strategy optimizes the model parameters to alleviate overfitting as much as possible in the second stage and improves prediction confidence. Our proposed method is validated on the public dataset LIDC-IDRI and the in-house dataset GDPHLUAD. Extensive experiments demonstrate that our method achieves promising performance and outperforms seven state-of-the-art semi-supervised models; Moreover, ablation experiments validate the effectiveness of various aspects of our approach.

Fault Feature Extraction Using L-Kurtosis and Minimum Entropy-Based Signal Demodulation

The health of mechanical components can be assessed by analyzing the vibration and acoustic signals they produce. These signals contain valuable information about the component’s condition, often encoded within specific frequency bands. However, extracting this information is challenging due to noise contamination from various sources. Narrow-band amplitude demodulation presents a robust technique for isolating fault-related information within the signal. This work proposes a novel approach based on cluster-based segmentation for demodulating the signal and extracting the frequency band of interest. The segmentation process leverages the criteria of maximum L-kurtosis and minimum entropy. L-kurtosis maximizes impulsiveness in the signal, while minimum entropy signifies a low degree of randomness and high cyclo-stationarity, and both characteristics are crucial for identifying the desired frequency band. Simulations and experimental tests using vibration signals from different gears demonstrate the effectiveness of this technique. The processed envelope of the signal exhibits distinct improvements, highlighting the ability to accurately extract the fault-related information embedded within the complex noise-ridden signals. This approach offers a promising solution for accurate and efficient fault diagnosis in mechanical systems, contributing to enhanced reliability and reduced downtime.

Exergy Flow as a Unifying Physical Quantity in Applying Dissipative Lagrangian Fluid Mechanics to Integrated Energy Systems.

Highly integrated energy systems are on the rise due to increasing global demand. To capture the underlying physics of such interdisciplinary systems, we need a modern framework that unifies all forms of energy. Here, we apply modified Lagrangian mechanics to the description of multi-energy systems. Based on the minimum entropy production principle, we revisit fluid mechanics in the presence of both mechanical and thermal dissipations and propose using exergy flow as the unifying Lagrangian across different forms of energy. We illustrate our theoretical framework by modeling a one-dimensional system with coupled electricity and heat. We map the exergy loss rate in real space and obtain the total exergy changes. Under steady-state conditions, our theory agrees with the traditional formula but incorporates more physical considerations such as viscous dissipation. The integral form of our theory also allows us to go beyond steady-state calculations and visualize the local, time-dependent exergy flow density everywhere in the system. Expandable to a wide range of applications, our theoretical framework provides the basis for developing versatile models in integrated energy systems.

Minimal zero entropy subshifts can be unrestricted along any sparse set

Abstract We present a streamlined proof of a result essentially presented by the author in [Some counterexamples in topological dynamics. Ergod. Th. & Dynam. Sys.28(4) (2008), 1291–1322], namely that for every set $S = \{s_1, s_2, \ldots \} \subset \mathbb {N}$ of zero Banach density and finite set A, there exists a minimal zero-entropy subshift $(X, \sigma )$ so that for every sequence $u \in A^{\mathbb {Z}}$ , there is $x_u \in X$ with $x_u(s_n) = u(n)$ for all $n \in \mathbb {N}$ . Informally, minimal deterministic sequences can achieve completely arbitrary behavior upon restriction to a set of zero Banach density. As a corollary, this provides counterexamples to the polynomial Sarnak conjecture reported by Eisner [A polynomial version of Sarnak’s conjecture. C. R. Math. Acad. Sci. Paris353(7) (2015), 569–572] which are significantly more general than some recently provided by Kanigowski, Lemańczyk and Radziwiłł [Prime number theorem for analytic skew products. Ann. of Math. (2)199 (2024), 591–705] and by Lian and Shi [A counter-example for polynomial version of Sarnak’s conjecture. Adv. Math.384 (2021), Paper no. 107765] and shows that no similar result can hold under only the assumptions of minimality and zero entropy.

Image based information hiding via minimization of entropy and randomization

In this paper, a new approach that can effectively and securely hide information into color images with significantly improved security and hiding capacity is proposed. The proposed approach performs information hiding in three major steps. As the first step, two binary sequences are constructed from the least significant bits in the pixels of a cover image and the information that needs to be embedded, the information entropies of both sequences are minimized with a dynamic programming method. In the second step, the resulting sequences are randomly reshuffled into randomized sequences with mappings based on a set of one-dimensional chaotic systems, a single binary sequence can be obtained by a matching operation performed between the two randomized sequences. Finally, an inverse mapping is applied to the sequence obtained in the second step, and the transformed sequence is embedded into the least significant bits in the pixels of the cover image. Both analysis and experiments show that the proposed approach can achieve guaranteed performance in both security and capacity for long binary sequences. In addition, a comparison with other state-of-the-art methods for image-based information hiding suggests that the proposed approach can achieve significantly improved performance and is promising for practical applications.

Principles of Molecular Evolution: Concepts from Non-equilibrium Thermodynamics for the Multilevel Theory of Learning.

We present a non-equilibrium thermodynamics approach to the multilevel theory of learning for the study of molecular evolution. This approach allows us to study the explicit time dependence of molecular evolutionary processes and their impact on entropy production. Interpreting the mathematical expressions, we can show that two main contributions affect entropy production of molecular evolution processes which can be identified as mutation and gene transfer effects. Accordingly, our results show that the optimal adaptation of organisms to external conditions in the context of evolutionary processes is driven by principles of minimum entropy production. Such results can also be interpreted as the basis of some previous postulates of the theory of learning. Although our macroscopic approach requires certain simplifications, it allows us to interpret molecular evolutionary processes using thermodynamic descriptions with reference to well-known biological processes.

Entropy optimization of a FENE-P viscoelastic model: a numerically guided comprehensive analysis

AbstractThe influence of polymers on entropy generation processes is substantial, particularly in the fields of fluid dynamics and rheology. The FENE-P (Finitely Extensible Nonlinear Elastic-Peterlin) model describes the polymer’s dynamics as a result of the interaction between the stretching caused by the velocity gradient and the elastic force that restores the polymer to its equilibrium position. Models such as FENE-P aid in understanding and predicting polymer flow behaviour allowing for the reduction of entropy generation by optimizing system designs. A continuum approach is employed to express the heat flux vector and polymer confirmation tensor of the model. The study investigates the complex relationship between polymer conformation, flow dynamics, and heat transfer taking into account the thermophoresis (Soret effect) and mass diffusion-thermal diffusion coupling (Dufour effect) phenomena to optimize processes by reducing entropy. This study illuminates polymer’s significance in entropy minimization, improving engineering design methodologies and applications in materials science, chemical engineering, and fluid dynamics. As result, the presence of polymers leads to a substantial decrease in the total entropy of the system. This understanding provides opportunities for enhancing heat transfer systems, thereby facilitating the development of more efficient and sustainable technology.

Entropy-guided robust feature domain adaptation for electroencephalogram-based cross-dataset drowsiness recognition

To accurately identify driver drowsiness using Electroencephalogram (EEG), a significant amount of time is typically required to gather data from either the same subject or multiple subjects under the same experimental condition. The prolonged period required for data collection impedes the practical application of EEG. In this paper, we consider the challenging task of cross-dataset driver drowsiness recognition, where the EEG data collected by different devices are expected to be conveniently recognized by a classifier trained on an existing dataset. In order to solve the problem of distribution drift, we propose an Entropy-Guided Robust Feature (EGRF) adaptation framework. This framework utilizes the Entropy Minimization (EM) technique for domain alignment and an ensemble-like structure, consisting of multiple branch classifiers and a dropout layer, to enhance the robustness. The proposed method has been tested across two public datasets and achieves 2-class recognition accuracies of 77.4% and 85.9%. The innovative feature-robust block, combined with the EM loss, leads to substantial performance improvement, proving the effectiveness of our framework for cross-dataset driver drowsiness recognition. Our research demonstrates a promising direction for development of a calibration-free driver drowsiness monitoring system in the future.

A Comparative Analysis of U-Net and Vision Transformer Architectures in Semi-Supervised Prostate Zonal Segmentation.

The precise segmentation of different regions of the prostate is crucial in the diagnosis and treatment of prostate-related diseases. However, the scarcity of labeled prostate data poses a challenge for the accurate segmentation of its different regions. We perform the segmentation of different regions of the prostate using U-Net- and Vision Transformer (ViT)-based architectures. We use five semi-supervised learning methods, including entropy minimization, cross pseudo-supervision, mean teacher, uncertainty-aware mean teacher (UAMT), and interpolation consistency training (ICT) to compare the results with the state-of-the-art prostate semi-supervised segmentation network uncertainty-aware temporal self-learning (UATS). The UAMT method improves the prostate segmentation accuracy and provides stable prostate region segmentation results. ICT plays a more stable role in the prostate region segmentation results, which provides strong support for the medical image segmentation task, and demonstrates the robustness of U-Net for medical image segmentation. UATS is still more applicable to the U-Net backbone and has a very significant effect on a positive prediction rate. However, the performance of ViT in combination with semi-supervision still requires further optimization. This comparative analysis applies various semi-supervised learning methods to prostate zonal segmentation. It guides future prostate segmentation developments and offers insights into utilizing limited labeled data in medical imaging.

A novel and fast electromagnetic and electrothermal software for quench analysis of high field magnets

Abstract High-field superconducting REBCO magnets contain several coils with many turns. For these magnets, electro-thermal quench is an issue that magnet designers need to take into account. Thus, there is a need for a fast and accurate software to numerically model the overall performance of full-scale magnets. High temperature superconductors can be modeled using different techniques for electro-magnetic and thermal (finite element method) analysis. However, it takes a lot of time to model the electro-magnetic and electro-thermal behavior of superconductors simultaneously, especially for non-insulated or metal-insulated coils. In addition, most of the available methods ignore screening currents, which are an important feature of REBCO magnets. We have developed a novel software programmed in C++, which performs coupled electro-magnetic and electro-thermal analysis using variational methods based on minimum electro-magnetic entropy production and finite difference, respectively. The developed software, which takes screening currents into account, is applied to axi-symmetric full scale magnets of more than 32 T field strength under the SuperEMFL project for thermal quench reliability during standard operation. We show that magnets incorporating non-insulated coils are more reliable against quench than metal insulated coils. Also, realistic cooling conditions at the boundaries are essential for such simulations. The model developed can be used for a quick and complete electro-magnetic and electro-thermal analysis of superconducting high field magnets.

Thermal Management and Entropy Minimization of Plain and Modified Shaped Plate Fin Heat Sinks Using Multi-Objective Genetic Algorithm

Thermal Management and Entropy Minimization of Plain and Modified Shaped Plate Fin Heat Sinks Using Multi-Objective Genetic Algorithm

An information-theoretic framework for conditional causality analysis of brain networks.

Identifying directed network models for multivariate time series is a ubiquitous problem in data science. Granger causality measure (GCM) and conditional GCM (cGCM) are widely used methods for identifying directed connections between time series. Both GCM and cGCM have frequency-domain formulations to characterize the dependence of time series in the spectral domain. However, the original methods were developed using a heuristic approach without rigorous theoretical explanations. To overcome the limitation, the minimum-entropy (ME) estimation approach was introduced in our previous work (Ning & Rathi, 2018) to generalize GCM and cGCM with more rigorous frequency-domain formulations. In this work, this information-theoretic framework is further generalized with three formulations for conditional causality analysis using techniques in control theory, such as state-space representations and spectral factorizations. The three conditional causal measures are developed based on different ME estimation procedures that are motivated by equivalent formulations of the classical minimum mean squared error estimation method. The relationship between the three formulations of conditional causality measures is analyzed theoretically. Their performance is evaluated using simulations and real neuroimaging data to analyze brain networks. The results show that the proposed methods provide more accurate network structures than the original approach.

Minimum entropy constrained cooperative inversion of electrical resistivity, seismic and magnetic data

Geophysical methods are widely used to gather information about the subsurface as they are non-intrusive and comparably cheap. However, the solution to the geophysical inverse problem is inherently non-unique, which introduces considerable uncertainties. As a partial remedy to this problem, independently acquired geophysical data sets can be jointly inverted to reduce ambiguities in the resulting multi-physical subsurface images. A novel cooperative inversion approach with joint minimum entropy constraints is used to create more consistent multi-physical images with sharper boundaries with respect to the single-method inversions. Here, this approach is implemented in an open-source software and its applicability on electrical resistivity tomography (ERT), seismic refraction tomography (SRT), and magnetic data is investigated. A synthetic 2D ERT and SRT data study is used to demonstrate the approach and to investigate the influence of the governing parameters. The findings showcase the advantage of the joint minimum entropy (JME) stabilizer over separate, conventional smoothness-constrained inversions. The method is then used to analyze field data from Rockeskyller Kopf, Germany. 3D ERT and magnetic data are combined and the results confirm the expected volcanic diatreme structure with improved details. The multi-physical images of both methods are consistent in some regions, as similar boundaries are produced in the resulting models. Because of its sensitivity to hydrologic conditions in the subsurface, observations suggest that the ERT method senses different structures than the magnetic method. These structures in the ERT result do not seem to be enforced on the magnetic susceptibility distribution, showcasing the flexibility of the approach. Both investigations outline the importance of a suitable parameter and reference model selection for the performance of the approach and suggest careful parameter tests prior to the joint inversion. With proper settings, the JME inversion is a promising tool for geophysical imaging, however, this work also identifies some objectives for future studies and additional research to explore and optimize the method.

Thermodynamic water adsorption analysis of biodegradable films based on corn starch-chitosan added with triblock copolymers

Adding triblock copolymers to hydrophilic films is an option to improve the functional properties and water resistance under high equilibrium relative humidity (RHeq) environments. Sorption isotherms of corn starch-chitosan based films added with triblock copolymers (L64 and P123) measured at 4, 20, and 36 °C were fitted using the Guggenheim-Anderson-De Boer equation. Thermodynamic analysis allowed allocating the minimum integral entropy value between 0.09 and 0.50 of water activity. The compensation enthalpy-entropy showed that the entropy driven the process at low water activities (0.10–∼0.33) and the enthalpy driven the water sorption at high water activities (∼0.33–∼0.90).

On the Occurrence of Domain Boundaries and Defects in Self-Organized Ordered Porous Anodic Alumina Systems.

The formation of domains and defects is a part of every self-organized structure without prepatterning which evolves from the electrochemical oxidation of valve metals like Ti, Al or Ta. Defects and domains, especially in highly ordered porous alumina films (PAOX), are one of the most studied phenomena in the current literature. The focus is on understanding their formation in order to find parameters to minimize their amount for applications that require the most highly organized structures. In this letter we give a solution to this problem by showing that the occurrence of defects and domains is an unavoidable consequence of the underlying self-organized process due to the simultaneous impact of the minimum (MINEP) and maximum (MAXEP) entropy production principle. As a consequence, experimental procedures to manufacture defect-free PAOX-films from a self-organization process face a fundamental limitation.

Simultaneous predictive bands for functional time series using minimum entropy sets

Functional Time Series (FTS) are sequences of dependent random elements taking values on some functional space. Most of the research on this domain focuses on producing a predictor able to forecast the next function, having observed a part of the sequence. For this, the Autoregressive Hilbertian process is a suitable framework. Here, we address the problem of constructing simultaneous predictive confidence bands for a stationary FTS. The method is based on an entropy measure for stochastic processes. To construct predictive bands, we use a Reproducing Kernel Hilbert Space (RKHS) to represent the functions and a functional bootstrap procedure that allows us to estimate the prediction law and a Reproducing Kernel Hilbert Spaces (RKHS) to represent the functions, considering then the basis associated to the reproducing kernel. We then classify the points on the projected space according to those that belong to the minimum entropy set (MES) and those that do not. We map the minimum entropy set back to the functional space and construct a band using the regularity property of the RKHS. The proposed methodology is illustrated through artificial and real data sets.

Fast Minimum Error Entropy for Linear Regression

The minimum error entropy (MEE) criterion finds extensive utility across diverse applications, particularly in contexts characterized by non-Gaussian noise. However, its computational demands are notable, and are primarily attributable to the double summation operation involved in calculating the probability density function (PDF) of the error. To address this, our study introduces a novel approach, termed the fast minimum error entropy (FMEE) algorithm, aimed at mitigating computational complexity through the utilization of polynomial expansions of the error PDF. Initially, the PDF approximation of a random variable is derived via the Gram–Charlier expansion. Subsequently, we proceed to ascertain and streamline the entropy of the random variable. Following this, the error entropy inherent to the linear regression model is delineated and expressed as a function of the regression coefficient vector. Lastly, leveraging the gradient descent algorithm, we compute the regression coefficient vector corresponding to the minimum error entropy. Theoretical scrutiny reveals that the time complexity of FMEE stands at O(n), in stark contrast to the O(n2) complexity associated with MEE. Experimentally, our findings underscore the remarkable efficiency gains afforded by FMEE, with time consumption registering less than 1‰ of that observed with MEE. Encouragingly, this efficiency leap is achieved without compromising accuracy, as evidenced by negligible differentials observed between the accuracies of FMEE and MEE. Furthermore, comprehensive regression experiments on real-world electric datasets in northwest China demonstrate that our FMEE outperforms baseline methods by a clear margin.