Source Entropy Research Articles

Colloidal particles as noise source for random number generation

In this work, we investigate colloidal particle patterns as a possible noise source for random number generation. We systematically analyze the minimum entropy of the noise source over different particle concentrations of {1, 3, 5, 7, 10, 12, 15} mg/ml according to the recommendations of the National Institute of Standards and Technology Special Publication 800-90B. The estimated minimum entropy of the non-independent and identically distributed particle pattern noise source is Hmin = 0.5896/1 bit at a particle amount of 5 mg/ml. For further entropy extraction on the noise source data, the secure hash algorithm is used to construct an entropy source. The randomness of the derived entropy source is verified according to the National Institute of Standards and Technology Special Publication 800-22 Rev. 1a and the accompanying statistical test suite. The entropy source passes all randomness tests of the statistical test suite and shows an estimated minimum entropy of Hmin = 0.9992/1 bit.

Impact of Shape Factor on a Couple Stress Flow of a Ternary Hybrid Nanofluid over an Inclined Flat Plate when Non-Uniform Heat Source and Thermal Radiation are Significant: A Darcy-Forchheimer Model

Abstract The study of fluid flow over an inclined flat plate finds applications in a diverse range of engineering fields including aerodynamics, energy production and automotive design. This study theoretically investigates the steady and radiative flow of a ternary hybrid nanofluid (Water + TiO2 + CoFe2O4 + MgO) with couple stress, using the Darcy-Forchheimer model. The flow occurs through a tilted flat plate and is subjected to irregular heat source parameter and entropy generation. The problem’s equations have been transformed into a collection of ordinary differential equations (ODEs), which has been skillfully resolved using the bvp4c solver. Graphs are utilized to elucidate outcomes for two instances of shape components, namely platelet and spherical. An escalation in the couple stress parameter (S) is demonstrated to be inversely related to the fluid velocity, resulting in a drop. Specifically, when 0.5 ≤ S ≤ 3, the friction factor exhibits a decline, with rates of 0.306201851 (for Platelet shape) and 0.304466755 (for Spherical shape). An intriguing observation reveals an augmentation in the generation of entropy as the volumetric fraction of TiO 2 rises. Upon investigation, it has been determined that when the Eckert number (Ecn) increases within the range of 0 ≤ Ecn ≤ 0.3, there is a significant reduction in the Nusselt number. Specifically, the decline is measured to be 0.328685192 for the platelet shape and 0.308939422 for the spherical shape. The utility of the Forchheimer number in regulating the fluid’s motion has been unveiled.

Capacity enhancement through entropy loading with probabilistically shaped signals in a frequency comb-based transmission system.

We demonstrate a multichannel entropy loading mechanism in an optical frequency comb-based coherent communication system. In high-capacity wavelength division multiplexing communications, the individual laser sources can be replaced by an optical frequency comb, thus reducing the complexity and energy consumption of the transmitter. However, the power variation among different comb lines will lead to performance discrepancies. Spectral flattening filters can be adopted to suppress the variation at the expense of an additional system loss. Alternatively, by applying probabilistic shaping, we have implemented multichannel entropy loading to facilitate a continuous adaptation of the source entropy. The data rate can be dynamically allocated according to the performance of each channel. Through the loading scheme, the non-uniform performances across the channels are effectively mitigated, achieving a capacity enhancement of 34.91 Gbit/s.

Entropic Bounds on the Average Length of Codes with a Space.

We consider the problem of constructing prefix-free codes in which a designated symbol, a space, can only appear at the end of codewords. We provide a linear-time algorithm to construct almost-optimal codes with this property, meaning that their average length differs from the minimum possible by at most one. We obtain our results by uncovering a relation between our class of codes and the class of one-to-one codes. Additionally, we derive upper and lower bounds to the average length of optimal prefix-free codes with a space in terms of the source entropy.

Method of information protection based on the integration of probabilistic encryption and noise immune coding

The subject matter of this article is the processes of increasing information security in communication systems to solve applied problems that are critical in the transmission of confidential data. The goal is to develop a multi-stage method for protecting information from unauthorized access, based on probabilistic encryption integration, error-correcting code, and error decorrelation. The tasks to be solved are as follows: to develop a probabilistic encryption algorithm that considers the entropy of the characters’ appearance in a message; to implement iterative coding with variable parameters of the information bits matrix; and to implement error decorrelation based on the generated matrix by interleaving information and check bits according to a given algorithm. Methodology: theory of signal communication and signal processing, theory of cryptography, theory of noise-resistant codes, probability theory, statistical analysis. The following results were obtained: a multi-stage method for protecting information from unauthorized access was proposed, in which at each step of data conversion, information secrecy is increased; probabilistic encryption was implemented, which considers the entropy of a discrete source of information when forming a space of random combinations; based on an iterative code with specified matrix parameters, an additional increase in the space of random combinations was implemented; error decorrelation reduces the multiplicity of errors in codewords and ensures mixing of bits according to a predetermined law. Conclusions. The scientific novelty of the results obtained is as follows: 1) an improved method of information protection from unauthorized access is proposed, based on probabilistic encryption integration, interference-resistant coding, and error decorrelation; 2) increasing information concealment is realized by eliminating the main shortcomings of probabilistic encryption, which did not take into account the entropy of the symbols of the open message; 3) it is proposed to form the space of random combinations taking into account the average probability of the appearance of a symbol in open texts, which will allow the output of a probabilistic cipher to form a stream of random combinations, which, according to its statistical properties, is close to the uniform distribution law; 4) further development of information protection methods based on interference-resistant coding and error decorrelation with encryption functions is received; 5) a further development of the statistical encryption method is the use of all redundant elements for both probabilistic encryption and interference-resistant coding.

Computer simulation of two phase power-law nanofluid of blood flow through a curved overlapping stenosed artery with induced magnetic field: entropy generation optimization

PurposeThe purpose of this study is to determine the impact of two-phase power law nanofluid on a curved arterial blood flow under the presence of ovelapped stenosis. Over the past couple of decades, the percentage of deaths associated with blood vessel diseases has risen sharply to nearly one third of all fatalities. For vascular disease to be stopped in its tracks, it is essential to understand the vascular geometry and blood flow within the artery. In recent scenarios, because of higher thermal properties and the ability to move across stenosis and tumor cells, nanoparticles are becoming a more common and effective approach in treating cardiovascular diseases and cancer cells.Design/methodology/approachThe present mathematical study investigates the blood flow behavior in the overlapped stenosed curved artery with cylinder shape catheter. The induced magnetic field and entropy generation for blood flow in the presence of a heat source, magnetic field and nanoparticle (Fe3O4) have been analyzed numerically. Blood is considered in artery as two-phases: core and plasma region. Power-law fluid has been considered for core region fluid, whereas Newtonian fluid is considered in the plasma region. Strongly implicit Stone’s method has been considered to solve the system of nonlinear partial differential equations (PDE’s) with 10–6 tolerance error.FindingsThe influence of various parameters has been discussed graphically. This study concludes that arterial curvature increases the probability of atherosclerosis deposition, while using an external heating source flow temperature and entropy production. In addition, if the thermal treatment procedure is carried out inside a magnetic field, it will aid in controlling blood flow velocity.Originality/valueThe findings of this computational analysis hold great significance for clinical researchers and biologists, as they offer the ability to anticipate the occurrence of endothelial cell injury and plaque accumulation in curved arteries with specific wall shear stress patterns. Consequently, these insights may contribute to the potential alleviation of the severity of these illnesses. Furthermore, the application of nanoparticles and external heat sources in the discipline of blood circulation has potential in the medically healing of illness conditions such as stenosis, cancer cells and muscular discomfort through the usage of beneficial effects.

Flow dynamics and noise generation mechanisms in supersonic underexpanded rectangular and planar jets

In this study, large eddy simulations of two rectangular jets with different aspect ratios and one planar jet with spanwise periodic boundaries are carried out in an attempt to investigate and clarify some key issues regarding the flow dynamics and noise generation mechanisms for the screech tone. The substantial effects of the nozzle configuration on shock structures and noise are first revealed in detail. It is found that when the lateral confinement is weakened, the spacing of the shock cells increases and the jet oscillates more intensively in the minor-axis plane, increasing the noise level and altering the screeching frequency. By analyzing the pressure fluctuations of the shear layer in the wavenumber-frequency space, different kinds of waves in these supersonic jets are examined. Importantly, it is revealed that the guided jet wave should play a dominant role in closing the resonance loops rather than the acoustic wave. Moreover, the energy-containing structures with pertinent frequencies are extracted by employing the reduced-order variational mode decomposition, and some underlying flow dynamics are presented. Especially, a novel mechanism has been identified: the low-frequency stretching motions of shock cells have a significant modulation effect on the screeching amplitude in the planar jet. Furthermore, with the aid of the multi-process acoustic theory, the characteristics of physical noise sources are diagnosed, particularly the source mechanisms related to the screech tone. The general structures and distribution of the kinematic noise source Sβ and entropy noise source Se are presented. Sβ mainly exhibits a vortex-like structure near the nozzle, while Se exhibits a lamellar bilayer structure. The spatiotemporal correlations between the physical sources and far-field noise show that the dominant mechanisms for the screech tone rely on the nozzle configuration and the screech tone tends to be produced by multiple sources in all cases.

Optimization of Protograph LDPC Codes via Surrogate Channel for Unequal Power Allocation

This paper investigates the possibilities of optimizing existing protograph low-density parity-check (PLDPC) codes and designing optimal PLDPC codes on both AWGN and Rayleigh channels under binary modulation with unequal power allocation (UPA) for nonuniform sources. While the binary UPA modulation achieves a power gain, it also results in unequal error protection on the coded bits in systematic codewords during transmission. For irregular codes, it is possible to best protect the information bits by optimizing the classification of variable nodes (VNs). Those VNs represent the coded bits in protograph that allows exploiting jointly the degree irregularity and the power differentiation. An algorithm for UPA-based protograph extrinsic information transfer (UPA-PEXIT) analysis on the Rayleigh channel is proposed with the concept of surrogate channel which circumvents the channel asymmetry problem with UPA. With the RJA code as example, it is shown that classification optimization does improve the decoding threshold when the source entropy is small. To design optimal codes, a UPA-based differential evolution (UPA-DE) algorithm is proposed in conjunction with the UPA-PEXIT analysis. A family of optimized codes are illustrated for three typical nonuniform source distributions and compared with the RJA and NND codes by simulations in error rate performance.

Rate-Exponent Region for a Class of Distributed Hypothesis Testing Against Conditional Independence Problems

We study a class of <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$K$ </tex-math></inline-formula> -encoder hypothesis testing against conditional independence problems. Under the criterion that stipulates minimization of the Type II error subject to a (constant) upper bound <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\epsilon $ </tex-math></inline-formula> on the Type I error, we characterize the set of encoding rates and exponent for both discrete memoryless and memoryless vector Gaussian settings. For the DM setting, we provide a converse proof and show that it is achieved using the Quantize-Bin-Test scheme of Rahman and Wagner. For the memoryless vector Gaussian setting, we develop a tight outer bound by means of a technique that relies on the de Bruijn identity and the properties of Fisher information. In particular, the result shows that for memoryless vector Gaussian sources the rate-exponent region is exhausted using the Quantize-Bin-Test scheme with Gaussian test channels; and there is no loss in performance caused by restricting the sensors’ encoders not to employ time sharing. Furthermore, we also study a variant of the problem in which the source, not necessarily Gaussian, has finite differential entropy and the sensors’ observations noises under the null hypothesis are Gaussian. For this model, our main result is an upper bound on the exponent-rate function. The bound is shown to mirror a corresponding explicit lower bound, except that the lower bound involves the source power (variance) whereas the upper bound has the source entropy power. Part of the utility of the established bound is for investigating asymptotic exponent/rates and losses incurred by distributed detection as function of the number of sensors.

Edge Learning for B5G Networks With Distributed Signal Processing: Semantic Communication, Edge Computing, and Wireless Sensing

To process and transfer large amounts of data in emerging wireless services, it has become increasingly appealing to exploit distributed data communication and learning. Specifically, edge learning (EL) enables local model training on geographically disperse edge nodes and minimizes the need for frequent data exchange. However, the current design of separating EL deployment and communication optimization does not yet reap the promised benefits of distributed signal processing, and sometimes suffers from excessive signalling overhead, long processing delay, and unstable learning convergence. In this paper, we provide an overview on practical distributed EL techniques and their interplay with advanced communication optimization designs. In particular, typical performance metrics for dual-functional learning and communication networks are discussed. Also, recent achievements of enabling techniques for the dual-functional design are surveyed with exemplifications from the mutual perspectives of "communications for learning" and "learning for communications." The application of EL techniques within a variety of future communication systems are also envisioned for beyond 5G (B5G) wireless networks. For the application in goal-oriented semantic communication, we present a first mathematical model of the goal-oriented source entropy as an optimization problem. In addition, from the viewpoint of information theory, we identify fundamental open problems of characterizing rate regions for communication networks supporting distributed learning-and-computing tasks. We also present technical challenges as well as emerging application opportunities in this field, with the aim of inspiring future research and promoting widespread developments of EL in B5G.

BINOMIAL METHODS FOR BINARY MESSAGE COMPRESSION

Two lossless numerical compression methods for binary messages using binomial coefficients are proposed. The methods are based on the decomposition of a Bernoulli source of information into a combinatorial source A and a probabilistic source B. Accordingly, the entropy of the source is also decomposed into two entropies. The entropy of source A solves the problem of enumeration encoding of compressible messages, and the entropy of source B for the first method solves the problem of optimal coding. The disadvantage of this method is the need for statistical tests. In the second method, the calculation of the maximum value of the entropy of the source B replaces the optimal encoding. This replacement avoids statistical tests, although it leads to a relatively small redundancy of compressed messages, which decreases as their length increases. The work of the methods is shown on specific examples, which allows you to better understand both the idea of the methods and their practical implementation.

ON COMBINATORIAL ENTROPY OF SOURCES OF BINARY INFORMATION

The paper considers the question of converting the entropy of a probabilistic information source generating binary sequences into combinatorial forms of its two constituent sources. One of them is basic, and the second has a conditional entropy with respect to the first. It is shown that together such sources make it possible to optimally encode information using numbering at the efficiency level of known methods of optimal coding. In terms of their functions, they are universal and can solve all the problems that are solved by the methods of coding probabilistic sources. However, unlike them, they do not require statistical tests before optimal coding, or they are carried out to a much lesser extent. They also simplify the calculation of the entropy of a probabilistic source of information, which is useful in solving many problems where it is necessary to know the value of the entropy of the source, for example, in error-correcting coding.

Data Deduplication With Random Substitutions

Data deduplication saves storage space by identifying and removing repeats in the data stream. Compared with traditional compression methods, data deduplication schemes are more time efficient and are thus widely used in large scale storage systems. In this paper, we provide an information-theoretic analysis on the performance of deduplication algorithms on data streams in which repeats are not exact. We introduce a source model in which probabilistic substitutions are considered. More precisely, each symbol in a repeated string is substituted with a given edit probability. Deduplication algorithms in both the fixed-length scheme and the variable-length scheme are studied. The fixed-length deduplication algorithm is shown to be unsuitable for the proposed source model as it does not take into account the edit probability. Two modifications are proposed and shown to have performances within a constant factor of optimal with the knowledge of source model parameters. We also study the conventional variable-length deduplication algorithm and show that as source entropy becomes smaller, the size of the compressed string vanishes relative to the length of the uncompressed string, leading to high compression ratios.

Spectrum Acquisition System Using Photoelectric Detection and Its Data Mining Simulation of Spectral Texture Features

Weak spectral signal detection technology is a new subject in the measurement engineering field. It obtains the characteristic information of the measured object based on the characteristics of light diffraction, interference, polarization and so on. Regarding light source entropy, the artificial light source of mercury argon lamp with a spectrum of 250 nm~918 nm is selected as the first research object. The grating monochromator with built-in grating scale lines (1200 g/mm, 600 g/mm) is selected as the light splitting device. Due to the weak spectral signal, the output signal is vulnerable to noise interference after a certain amount of photoelectric conversion. AD202 variable voltage coupling filtering technology is adopted. It means that in the signal isolation and amplification process, the AD202 variable voltage coupled filter amplifier chip is employed to design the current/voltage conversion circuit and the same direction proportional operation amplifier circuit. Meanwhile, a second-order low-pass filter circuit is designed based on the characteristics of the OP07 operational amplifier. For the data mining of multispectral image texture features, Contourlet transform is used, that is, the multispectral image texture features are fuzzy preprocessed to remove the adjacent range correlation knowledge. On this basis, the texture features of multispectral images are analyzed. In the experiment, Contourlet transform is adopted to transform multispectral images from spatial domain to frequency domain, which has the advantages of short extraction time, low cost, and high mining accuracy. A spectrum acquisition system based on photoelectric detection is designed. With chlorophyll as the research object, the sample solution to be tested has high photoelectric absorbance in the range of blue-violet light (400 nm~500 nm) and red-orange light (600 nm~700 nm).

Unsupervised online anomaly detection in Software Defined Network environments

Software Defined Networking (SDN) simplifies network management and significantly reduces operational costs. SDN removes the control plane from forwarding devices (e.g., routers and switches) and centralizes this plane in a controller, enabling the management of the network forwarding decisions by programming the control plane with a high-level language. However, its centralized architecture may be compromised by flooding attacks, such as Distributed Denial of Service (DDoS) and portscan. Facing this challenge, we propose an Intrusion Detection System (IDS) based on online clustering to detect attacks in an evolving SDN network taking advantage of the entropy of source and destination IP addresses and ports. Our proposal is focused on avoiding the demand for labeling and previous knowledge to provide a practical and accurate method to address real-life online scenarios. Moreover, our proposal paves the way for a comprehensive analysis by projecting the cluster’s structure over the feature space, providing insights on intensity, seasonality, and attack type. Our experiments were carried out with the DenStream algorithm in several databases attacked by DDoS and portscan with different intensities, durations, and overlapping patterns. When comparing DenStream performance to Half-Space-Trees, an accurate online one-class classification algorithm for anomaly detection, it was possible to expose the capacity of our unsupervised proposal, overcoming the one-class solution, and reaching f-measure rates above 99.60%.

On the Information Theory for Magnetic Resonance Imaging.

This work presents the mathematical formulation of the Magnetic Resonance Imaging (MRI) system modeled as a wireless communication system to establish its information theory foundations. The MRI system conceived as a source-sink communication system has channel impairments that affect the transmitted data. The information source is a stochastic process that produces a sequence of information symbols governed by a set of probabilities. The adverse effects on the transmitted MRI signal shall limit the amount of information capable of being received at the sink. Therefore, reliable detection at the receiver shall be accomplished by estimating the channel capacity and an approximation of the source entropy. Modeling the MRI system using a wireless model shall simplify the receiver architecture, yielding new methods to improve MRI signal acquisition, i.e., different values of bandwidth and signal strength yield the same channel capacity. Achieving capacity bridges information and computation efficiency.Clinical relevance This work establishes the basics to reduce MRI scan times by acquiring sufficient information with less redundant information.

Measuring chaos in the Lorenz and Rössler models: Fidelity tests for reservoir computing.

This study focuses on the qualitative and quantitative characterization of chaotic systems with the use of a symbolic description. We consider two famous systems, Lorenz and Rössler models with their iconic attractors, and demonstrate that with adequately chosen symbolic partition, three measures of complexity, such as the Shannon source entropy, the Lempel-Ziv complexity, and the Markov transition matrix, work remarkably well for characterizing the degree of chaoticity and precise detecting stability windows in the parameter space. The second message of this study is to showcase the utility of symbolic dynamics with the introduction of a fidelity test for reservoir computing for simulating the properties of the chaos in both models' replicas. The results of these measures are validated by the comparison approach based on one-dimensional return maps and the complexity measures.

Information in morphological characters.

The construction of morphological character matrices is central to paleontological systematic study, which extracts paleontological information from fossils. Although the word information has been repeatedly mentioned in a wide array of paleontological systematic studies, its meaning has rarely been clarified nor specifically defined. It is important, however, to establish a standard to measure paleontological information because fossils are hardly complete, rendering the recognition of homologous and homoplastic structures difficult. Here, based on information theory, we show the deep connections between paleontological systematic study and communication system engineering. Information is defined as the decrease of uncertainty and it is the information in morphological characters that allows distinguishing operational taxonomic units (OTUs) and reconstructing evolutionary history. We propose that concepts in communication system engineering such as source coding and channel coding, correspond to the construction of diagnostic features and the entire character matrices in paleontological studies. The two coding strategies should be distinguished following typical communication system engineering, because they serve dual purposes. With character matrices from six different vertebrate groups, we analyzed their information properties including source entropy, mutual information, and channel capacity. Estimation of channel capacity shows character saturation of all matrices in transmitting paleontological information, indicating that, due to the presence of noise, oversampling characters not only increases the burden in character scoring, but also may decrease quality of matrices. We further test the use of information entropy, which measures how informative a variable is, as a character weighting criterion in parsimony‐based systematic studies. The results show high consistency with existing knowledge with both good resolution and interpretability.

Selecting an Effective Entropy Estimator for Short Sequences of Bits and Bytes with Maximum Entropy.

Entropy makes it possible to measure the uncertainty about an information source from the distribution of its output symbols. It is known that the maximum Shannon’s entropy of a discrete source of information is reached when its symbols follow a Uniform distribution. In cryptography, these sources have great applications since they allow for the highest security standards to be reached. In this work, the most effective estimator is selected to estimate entropy in short samples of bytes and bits with maximum entropy. For this, 18 estimators were compared. Results concerning the comparisons published in the literature between these estimators are discussed. The most suitable estimator is determined experimentally, based on its bias, the mean square error short samples of bytes and bits.

Accelerated implementation for testing IID assumption of NIST SP 800-90B using GPU.

In cryptosystems and cryptographic modules, insufficient entropy of the noise sources that serve as the input into random number generator (RNG) may cause serious damage, such as compromising private keys. Therefore, it is necessary to estimate the entropy of the noise source as precisely as possible. The National Institute of Standards and Technology (NIST) published a standard document known as Special Publication (SP) 800-90B, which describes the method for estimating the entropy of the noise source that is the input into an RNG. The NIST offers two programs for running the entropy estimation process of SP 800-90B, which are written in Python and C++. The running time for estimating the entropy is more than one hour for each noise source. An RNG tends to use several noise sources in each operating system supported, and the noise sources are affected by the environment. Therefore, the NIST program should be run several times to analyze the security of RNG. The NIST estimation runtimes are a burden for developers as well as evaluators working for the Cryptographic Module Validation Program. In this study, we propose a GPU-based parallel implementation of the most time-consuming part of the entropy estimation, namely the independent and identically distributed (IID) assumption testing process. To achieve maximal GPU performance, we propose a scalable method that adjusts the optimal size of the global memory allocations depending on GPU capability and balances the workload between streaming multiprocessors. Our GPU-based implementation excluded one statistical test, which is not suitable for GPU implementation. We propose a hybrid CPU/GPU implementation that consists of our GPU-based program and the excluded statistical test that runs using OpenMP. The experimental results demonstrate that our method is about 3 to 25 times faster than that of the NIST package.