Principles Of Information Theory Research Articles

A Novel Hypersonic Target Trajectory Estimation Method Based on Long Short-Term Memory and a Multi-Head Attention Mechanism.

To address the complex maneuvering characteristics of hypersonic targets in adjacent space, this paper proposes an LSTM trajectory estimation method combined with the attention mechanism and optimizes the model from the information-theoretic perspective. The method captures the target dynamics by using the temporal processing capability of LSTM, and at the same time improves the efficiency of information utilization through the attention mechanism to achieve accurate prediction. First, a target dynamics model is constructed to clarify the motion behavior parameters. Subsequently, an LSTM model incorporating the attention mechanism is designed, which enables the model to automatically focus on key information fragments in the historical trajectory. In model training, information redundancy is reduced, and information validity is improved through feature selection and data preprocessing. Eventually, the model achieves accurate prediction of hypersonic target trajectories with limited computational resources. The experimental results show that the method performs well in complex dynamic environments with improved prediction accuracy and robustness, reflecting the potential of information theory principles in optimizing the trajectory prediction model.

A dynamic symmetric key generation at wireless link layer: information-theoretic perspectives

The expansion of wireless communication introduces security vulnerabilities, emphasizing the essential need for secure systems that prioritize confidentiality, integrity, and other key aspects of data protection. Since computational security acknowledges the possibility of breaches when adequate computational resources are available, that is why information-theoretic security is being explored, which suggests the existence of unbreakable cryptographic systems even in the presence of limitless processing power. Secret key exchange has traditionally relied on RSA or DH protocols, but researchers are now exploring innovative approaches for sharing secret keys among wireless network devices, leveraging physical or link layer characteristics. This research seeks to revolutionize secure multi-party key acquisition in wireless networks, capitalizing on information-theoretic security and collaborative data extraction. The proposed secret key generation framework comprehensively organizes and explains the information-theoretic aspects of secret key generation within the lower layers of wireless networks, especially the link layer, proposes a novel information-theoretic SKG framework for the dynamic acquisition of symmetric secret keys, and responds to contemporary information security challenges by relying on information-theory principles rather than vulnerable mathematical relationships in the post-quantum period. A new cryptographic key can be generated using a straightforward method, and when it is combined (XORed) with the previous key, it creates a continuously changing secret for encryption and decryption. This approach enhances security because, as attackers attempt to break the encryption, the system generates fresh, dynamic keys, making it progressively more challenging for them to succeed. The research work in question integrates key renewal, or how often keys are updated (dynamic keys), with a security off-period. It introduces a framework for determining the best key refresh rate based on the anticipated rate at which keys might be compromised. Furthermore, the proposed framework is scalable, allowing new nodes to quickly join the existing network. The system was tested with multiple nodes equipped with IEEE 802.11 interfaces, which were set in monitor mode to capture frames at the link layer. Nodes map their on-time frames onto their Bloom filters. Nodes exchange these Bloom filters in a feedback mechanism. Nodes extract those frames from their .pcap files, which are present in all Bloom filters; these are common frames among all nodes. These frames are used to form a shared secret that is passed to HMAC Key Derivation Function by each node to acquire the final encryption key of the required length. The validation of this encryption key is performed using a simple challenge-response protocol; upon successful validation, encrypted communication begins. Otherwise, the key generation process is restarted.

Quantum entropy couples matter with geometry

Abstract We propose a theory for coupling matter fields with discrete geometry on higher-order networks, i.e. cell complexes. The key idea of the approach is to associate to a higher-order network the quantum entropy of its metric. Specifically we propose an action having two contributions. The first contribution is proportional to the logarithm of the volume associated to the higher-order network by the metric. In the vacuum this contribution determines the entropy of the geometry. The second contribution is the quantum relative entropy between the metric of the higher-order network and the metric induced by the matter and gauge fields. The induced metric is defined in terms of the topological spinors and the discrete Dirac operators. The topological spinors, defined on nodes, edges and higher-dimensional cells, encode for the matter fields. The discrete Dirac operators act on topological spinors, and depend on the metric of the higher-order network as well as on the gauge fields via a discrete version of the minimal substitution. We derive the coupled dynamical equations for the metric, the matter and the gauge fields, providing an information theory principle to obtain the field theory equations in discrete curved space.

Infrared and Harsh Light Visible Image Fusion Using an Environmental Light Perception Network.

The complementary combination of emphasizing target objects in infrared images and rich texture details in visible images can effectively enhance the information entropy of fused images, thereby providing substantial assistance for downstream composite high-level vision tasks, such as nighttime vehicle intelligent driving. However, mainstream fusion algorithms lack specific research on the contradiction between the low information entropy and high pixel intensity of visible images under harsh light nighttime road environments. As a result, fusion algorithms that perform well in normal conditions can only produce low information entropy fusion images similar to the information distribution of visible images under harsh light interference. In response to these problems, we designed an image fusion network resilient to harsh light environment interference, incorporating entropy and information theory principles to enhance robustness and information retention. Specifically, an edge feature extraction module was designed to extract key edge features of salient targets to optimize fusion information entropy. Additionally, a harsh light environment aware (HLEA) module was proposed to avoid the decrease in fusion image quality caused by the contradiction between low information entropy and high pixel intensity based on the information distribution characteristics of harsh light visible images. Finally, an edge-guided hierarchical fusion (EGHF) module was designed to achieve robust feature fusion, minimizing irrelevant noise entropy and maximizing useful information entropy. Extensive experiments demonstrate that, compared to other advanced algorithms, the method proposed fusion results contain more useful information and have significant advantages in high-level vision tasks under harsh nighttime lighting conditions.

Information poetics: communication and combinatorics in concrete poetry

ABSTRACT While Max Bense advocated information aesthetics primarily in his theoretical works, many Concrete poets applied the principles of information theory and modern communications technologies to their poetry, whether or not they explicitly embraced Bense’s analytical framework. As Eugen Gomringer (the ‘father’ of German-language Concrete poetry) wrote in his manifesto ‘from line to constellation’: ‘our languages are in the process of formal simplification. reduced, terse forms are emerging’. But Concrete poetry was notable not only for its concision, or for the visual and spatial emphases generally said to characterise it; rather, many Concrete poets also employed algorithmic or combinatoric methods or used repetition and variation in ways that mirrored the functions of redundancy and entropy in Claude Shannon’s theory of communication. With Bense’s information aesthetics as background, this essay will explore how the concept of information shaped the practice of these Concrete poets.

Unraveling schizophrenia's genetic complexity through advanced causal inference and chromatin 3D conformation

Schizophrenia is a polygenic complex disease with a heritability as high as 80 %, yet the mechanism of polygenic interaction in its pathogenesis remains unclear. Studying the interaction and regulation of schizophrenia susceptibility genes is crucial for unraveling the pathogenesis of schizophrenia and developing antipsychotic drugs. Therefore, we developed a bioinformatics method named GRACI (Gene Regulation Analysis based on Causal Inference) based on the principles of information theory, a causal inference model, and high order chromatin 3D conformation. GRACI captures the interaction and regulatory relationships between schizophrenia susceptibility genes by analyzing genotyping data. Two datasets, comprising 1459 and 2065 samples respectively, were analyzed, and the gene networks from both datasets were constructed. GRACI showcased superior accuracy when compared to widely adopted methods for detecting gene-gene interactions and intergenic regulation. This alignment was further substantiated by its correlation with chromatin high-order conformation patterns. Using GRACI, we identified three potential genes—KCNN3, KCNH1, and KCND3—that are directly associated with schizophrenia pathogenesis. Furthermore, the results of GRACI on the standalone dataset illustrated the method's applicability to other complex diseases. GRACI download: https://github.com/liuliangjie19/GRACI

Is Biofuel HVO An Alternative In Aviation? An Infodynamic Comparative Analysis

The transition from fossil fuels to sustainable alternatives is imperative. This research evaluates Hydroprocessed Vegetable Oil (HVO) as a substitute for Jet A-1 in aviation. Focusing on droplet sizes in fuel sprays, which are critical for optimizing combustion, we explore an informational perspective for comparative analysis. Integrating information theory principles, we introduce the terms informature, infotropy, and infosensor that quantify and capture the nondeterministic nature of physical systems. Our analysis reveals similar drop size distributions for HVO and Jet A-1, with the Gamma function effectively characterizing the distributions. Both fuels exhibit spray complexity evolving toward higher states, emphasizing the role of aerodynamic forces and minimum development distance in atomization. We propose a novel lexicon, infodynamics, viewing sprays as networks of information flows, with infotropy providing context-rich insights. HVO is endorsed as a viable alternative, with broader implications for sustainable aviation solutions and understanding complex engineering processes.

Entropy, melody, beauty: composing with information theory

ABSTRACT Within a few years of information theory’s popularisation through the writings of Claude Shannon and Norbert Wiener, its basic framework was adopted and adapted by a loose network of music theorists, composers, and aestheticians, for whom a core principle of information theory – that ‘the aesthetic content of music can be treated in terms of fluctuations between the two extremes of total randomness and total redundancy’ (Hiller and Isaacson 1959, 110) – became an article of faith. Although it was sometimes imagined to offer a neutral conceptual framework for thinking about artmaking or a scientific alternative to the vagueness of previous artistic discourses, information theory was necessarily embroiled in the aesthetic debates of its time. In this paper I examine the encounter between information theory and musical composition in the middle of the twentieth century, paying special attention to the question of how the reception of information-theoretical concepts among composers and theorists inflected pre-existing debates about the nature and function of music in modernity.

Data storage format for analytical systems based on metadata and dependency graphs between CSV and JSON

In the modern information society, the volume of data is constantly growing, and its effective processing is becoming key for enterprises. The transmission and storage of this data also plays a critical role. Big data used in analytics systems is most often transmitted in one of two popular formats: CSV for structured data and JSON for unstructured data. However, existing file formats may not be effective or flexible enough for certain data analysis tasks. For example, they may not support complex data structures or provide sufficient control over metadata. Alternatively, analytical tasks may require additional information about the data, such as metadata, data schema, etc. Based on the above, the subject of this study is a data format based on the combined use of CSV and JSON for processing and analyzing large amounts of information. The option of sharing the designated data types for the implementation of a new data format is proposed. For this purpose, designations have been introduced for the data structure, which includes CSV files, JSON files, metadata and a dependency graph. Various types of functions are described, such as aggregating, transforming, filtering, etc. Examples of the application of these functions to data are given. The proposed approach is a technique that can significantly facilitate the processes of information analysis and processing. It is based on a formalized approach that allows you to establish clear rules and procedures for working with data, which contributes to their more efficient processing. Another aspect of the proposed approach is to determine the criteria for choosing the most appropriate data storage format. This criterion is based on the mathematical principles of information theory and entropy. The introduction of a criterion for choosing a data format based on entropy makes it possible to evaluate the information content and compactness of the data. This approach is based on the calculation of entropy for selected formats and weights reflecting the importance of each data value. By comparing entropies, you can determine the required data transmission format. This approach takes into account not only the compactness of the data, but also the context of their use, as well as the possibility of including additional meta-information in the files themselves and supporting data ready for analysis.

Researching brain-computer interfaces for enhancing communication and control in neurological disorders

The paper provides a comprehensive exploration of brain-computer interfaces (BCIs) and their application in addressing communication and control challenges in neurological disorders like Amyotrophic Lateral Sclerosis (ALS), locked-in syndrome, Parkinson’s disease, and Spinal Cord Injury. It traces BCI advancements from foundational information theory principles to their current state. Neurological disorders severely impact communication and control abilities, with ALS causing muscle weakness and paralysis, and locked-in syndrome confining individuals within paralyzed bodies while maintaining cognitive functions. BCIs decode brain signals, enabling control of external devices like computers, offering hope for restoring communication and control in affected individuals. Employing data analysis and visualization techniques, this study evaluates BCI performance in improving communication and control across various disorders. Matplotlib generates informative graphs and performance metrics, quantifying BCI efficacy for users with differing motor impairments. The findings highlight BCI transformative potential, guiding clinicians and researchers toward personalized solutions for diverse patient populations. This research underscores the necessity for continued innovation and exploration in BCI technology, envisioning a more inclusive and adaptive future for individuals with neurological disorders

Information system for assessing the informativeness of an epidemic process features

The primary objective of this study is to assess the informativeness of various parameters influencing epidemic processes utilizing the Shannon and Kullback–Leibler methods. These methods were selected based on their foundation in the principles of information theory and their extensive application in machine learning, statistics, and other relevant domains. A comparative analysis was performed between the results acquired from both methods, and an information system was designed to facilitate the uploading of data samples and the calculation of factor informativeness impacting the epidemic processes. The findings revealed that certain features, such as “Chronic lung disease,” “Chronic kidney disease,” and “Weakened immunity,” did not carry significant information for further analysis and hindered the forecasting process, as per the data set examined. The developed information system efficiently supports the assessment of feature informativeness, thereby aiding in the comprehensive analysis of epidemic processes and enabling the visualization of the results. This study contributes to the current body of knowledge by providing specific examples of applying the described algorithmic models, comparing various methods and their outcomes, and developing a supportive tool for analyzing epidemic processes.

Analyzing Polytomous Test Data: A Comparison Between an Information-Based IRT Model and the Generalized Partial Credit Model

Item response theory (IRT) models the relationship between the possible scores on a test item against a test taker’s attainment of the latent trait that the item is intended to measure. In this study, we compare two models for tests with polytomously scored items: the optimal scoring (OS) model, a nonparametric IRT model based on the principles of information theory, and the generalized partial credit (GPC) model, a widely used parametric alternative. We evaluate these models using both simulated and real test data. In the real data examples, the OS model demonstrates superior model fit compared to the GPC model across all analyzed datasets. In our simulation study, the OS model outperforms the GPC model in terms of bias, but at the cost of larger standard errors for the probabilities along the estimated item response functions. Furthermore, we illustrate how surprisal arc length, an IRT scale invariant measure of ability with metric properties, can be used to put scores from vastly different types of IRT models on a common scale. We also demonstrate how arc length can be a viable alternative to sum scores for scoring test takers.

Basic uncertainty information hesitant fuzzy multi-attribute decision-making method with credibility

This study addresses the inadequacy of the current quantitative calculation method for decision-maker credibility in hesitant fuzzy multi-attribute decision-making, where credibility is considered. To overcome this limitation, a novel quantitative calculation method for decision-maker credibility is proposed based on the principles of basic uncertainty information theory under a hesitant fuzzy environment. Furthermore, a credible-based hesitant fuzzy multi-attribute decision model is developed. Initially, the paper introduces the concept of a basic uncertainty hesitant fuzzy set by combining basic uncertainty information theory with hesitant fuzzy set theory, thereby enhancing the understanding of basic uncertainty information theory within the realm of non-interval fuzzy information. Building on this foundation, the method for determining the hesitant degree of each element in the basic uncertainty hesitant fuzzy set is provided, followed by the proposed quantitative calculation method for decision-maker’s credibility under the hesitant fuzzy environment, which addresses the lack of a quantitative approach for assessing expert credibility under such circumstances. Subsequently, an attribute weight assignment method is introduced, considering the decision-maker’s credibility, leading to the formulation of a basic uncertainty hesitant fuzzy multi-attribute decision model based on credibility. This model enhances existing hesitant fuzzy multi-attribute decision-making methods that take credibility into account. To validate the proposed approach, the study applies it to the selection of new energy vehicle battery suppliers. The results of the analysis using actual data and sensitivity analysis demonstrate that decision-maker credibility can be quantitatively determined using the proposed method. Additionally, the basic uncertainty hesitant fuzzy multi-attribute decision-making model based on credibility effectively aids in supplier selection. The feasibility and stability of this method are verified through the examination of risk appetite coefficient and hesitancy coefficient.

Analysis of the packaging characteristics of creative products in the context of new media based on Bayesian networks

Abstract Clarifying the features of the packaging of cultural and creative products is beneficial to out-of-the-loop cultural and creative products. In this paper, starting from probability theory, graph theory and information theory principles, we combine Bayesian network structure and feature selection theory to construct a feature extraction model for cultural and creative product packaging. The maximum mutual information principle is used to optimize the feature selection scoring function, and experimental analyses of execution time and performance comparisons are conducted for the BNCC-FS method. In terms of execution time, the average training time of the BNCC-FS method on the eight datasets is 4.73s, which is 14.53% and 43.44% lower compared to CC-FS and ECC-FS, respectively. In terms of performance comparison, the average HammingLoss value of the BNCC-FS method on sixteen datasets was 0.155, which was 2.52% and 10.41% lower compared to CC-FS and ECC-FS, respectively. This shows that the Bayesian network-based feature extraction method can effectively achieve feature extraction of the data and also provides a new research database for the analysis of the packaging features of cultural and creative products.

A discriminative information-theoretical analysis of the regularity gradient in inflectional morphology

Over the last decades, several independent lines of research in morphology have questioned the hypothesis of a direct correspondence between sublexical units and their mental correlates. Word and paradigm models of morphology shifted the fundamental part-whole relation in an inflection system onto the relation between individual inflected word forms and inflectional paradigms. In turn, the use of artificial neural networks of densely interconnected parallel processing nodes for morphology learning marked a radical departure from a morpheme-based view of the mental lexicon. Lately, in computational models of Discriminative Learning, a network architecture has been combined with an uncertainty reducing mechanism that dispenses with the need for a one-to-one association between formal contrasts and meanings, leading to the dissolution of a discrete notion of the morpheme.The paper capitalises on these converging lines of development to offer a unifying information-theoretical, simulation-based analysis of the costs incurred in processing (ir)regularly inflected forms belonging to the verb systems of English, German, French, Spanish and Italian. Using Temporal Self-Organising Maps as a computational model of lexical storage and access, we show that a discriminative, recurrent neural network, based on Rescorla-Wagner’s equations, can replicate speakers’ exquisite sensitivity to widespread effects of word frequency, paradigm entropy and morphological (ir)regularity in lexical processing. The evidence suggests an explanatory hypothesis linking Word and paradigm morphology with principles of information theory and human perception of morphological structure. According to this hypothesis, the ways more or less regularly inflected words are structured in the mental lexicon are more related to a reduction in processing uncertainty and maximisation of predictive efficiency than to economy of storage.

Robust model estimation by using preference analysis and information theory principles

Robust model estimation by using preference analysis and information theory principles

Designing Security for the Sixth Generation: About Necessity, Concepts and Opportunities

Intelligent, comprehensive and, above all, secure wireless interconnection is the driving force behind technological progress. To ensure this, the development towards Sixth Generation (6G) Wireless Systems has been launched and is scheduled to be operational by 2030. This data technology of the future turns 6G into the infrastructure of a new generation of mobile, intelligent, and context-sensitive services, available everywhere and featuring high trustworthiness and performance, relying on both, network-side and off-network context sources. In addition, the networks themselves ought to become intelligent and thus more efficient and resource-saving, which requires a high degree of automated utilization of Artificial Intelligence (AI). Building upon the principles of information and communication theory for both the physical (bit)-transmission layer (PHY) and media access, new communication concepts for 6G will be developed providing the foundations for research into new single and multi-user operation, access and core networks. The flip side of this coin of opportunities: Sophisticated technology inevitably leads to additional security vulnerabilities, open access systems and Open-Radio Access Network (O-RAN) approaches imply new attack vectors. The holistic interconnection of everything renders it ever more attractive to attackers to harm systems, and create damage. Furthermore, enhanced computational power along with quantum computers make conventional systems more vulnerable than ever, and the value of the transmitted data increases tremendously: It is not only machine and sensor data, but also very personal and healthcare data transmitted with 6G. Therefore, the aim is to build a resilient and secure 6G system capable of recognizing attacks and uncertainties, flexibly absorbing them, recovering in a timely and sustainable manner, and compensating for impaired functionality through transformation. This holistic resilience-by-design approach is based, among other things, on technology such as Quantum Key Distribution (QKD) and Post Quantum-Crypto to achieve end-to-end security, Reconfigurable Intelligent Surfaces (RISs) to rely, control and manipulate the wireless transmission channel, Wireless Optical Communication (WOC), Physical Layer Security (PhySec), but also Body Area Networks (BANs), the integration of the human body relying on biometrics and the Tactile Internet (TI). These concepts will be discussed and shed light on in the scope of this work.

A complete method for assessing the effectiveness of eyewitness identification procedures: Expected information gain.

We present a method for measuring the efficacy of eyewitness identification procedures by applying fundamental principles of information theory. The resulting measure evaluates the expected information gain (EIG) for an identification attempt, a single value that summarizes an identification procedure's overall potential for reducing uncertainty about guilt or innocence across all possible witness responses. In a series of demonstrations, we show that EIG often disagrees with existing measures (e.g., diagnosticity ratios or area under the receiver operating characteristic) about the relative effectiveness of different identification procedures. Each demonstration is designed to highlight key distinctions between existing measures and EIG. An overarching theme is that EIG provides a complete measure of evidentiary value, in the sense that it factors in all aspects of identification performance. Collectively, these demonstrations show that EIG has substantial potential to inspire new discoveries in eyewitness research and provide a new perspective on policy recommendations for the use of identifications in real investigations. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

Informed decision-making: Statistical methodology for surrogacy evaluation and its role in licensing and reimbursement assessments.

The desire, by patients and society, for faster access to therapies has driven a long tradition of the use of surrogate endpoints in the evaluation of pharmaceuticals and, more recently, biologics and other innovative medical technologies. The consequent need for statistical validation of potential surrogate outcome measures is a prime example on the theme of statistical support for decision‐making in health technology assessment (HTA). Following the pioneering methodology based on hypothesis testing that Prentice presented in 1989, a host of further methods, both frequentist and Bayesian, have been developed to enable the value of a putative surrogate outcome to be determined. This rich methodological seam has generated practical methods for surrogate evaluation, the most recent of which are based on the principles of information theory and bring together ideas from the causal effects and causal association paradigms. Following our synopsis of statistical methods, we then consider how regulatory authorities (on licensing) and payer and HTA agencies (on reimbursement) use clinical trial evidence based on surrogate outcomes. We review existing HTA surrogate outcome evaluative frameworks. We conclude with recommendations for further steps: (1) prioritisation by regulators and payers of the application of formal surrogate outcome evaluative frameworks, (2) application of formal Bayesian decision‐analytic methods to support reimbursement decisions, and (3) greater utilization of conditional surrogate‐based licensing and reimbursement approvals, with subsequent reassessment of treatments in confirmatory trials based on final patient‐relevant outcomes.

Experimental protocol for testing the mass–energy–information equivalence principle

The mass–energy–information equivalence principle proposed in 2019 and the information content of the observable matter in the universe estimated in 2021 represent two important conjectures, called the information conjectures. Combining information theory and physical principles of thermodynamics, these theoretical proposals made specific predictions about the mass of information as well as the most probable information content per elementary particle. Here, we propose an experimental protocol that allows for empirical verification of the information conjectures by confirming the predicted information content of elementary particles. The experiment involves a matter–antimatter annihilation process. When an electron–positron annihilates, in addition to the two 511 keV gamma photons resulting from the conversion of their rest masses into energy, we predict that two additional low energy photons should be detected, resulting from their information content erasure. At room temperature, a positron–electron annihilation should produce two ∼50 µm wavelength infrared photons due to the information erasure. This experiment could, therefore, confirm both information conjectures and the existence of information as the fifth state of matter in the universe.