Foundations and Trends® in Communications and Information Theory

: Calculating the amount of information about a random function contained in another random function has important uses in communication theory. An expression for the mutual information for continuous time random processes has been given by Gelfand and Yaglom, Chiang, and Perez by generalizing Shannon's result in a natural way. Under a condition of absolute continuity of measures the continuous time expression has the same form as Shannon's result. For two Gaussian processes Gelfand and Yaglom express the mutual information in terms of a mean square estimation error. We generalize this result to diffusion processes and express the solution in a different form which is more naturally related to a corresponding filtering problem. We also use these results to calculate some information rates.

Perfect Codes in the Lee Metric and the Packing of Polyominoes

Gaussian channels capacity, studying supremum of information transmission rates with small error probability

The information really useful to humans must be the trinity comprising three aspects: the form termed syntactic information, the meaning termed semantic information, and the use termed pragmatic information. The most well-known theory of information proposed by Shannon in 1948, often considered to be “the” theory of information is a statistical theory of syntactic information. Thus, semantic and pragmatic aspects are not addressed by Shannon. Instead they are fixed to the process of communication of sender to receiver via noisy channel. In this paper a theory of semantic information will be presented. It will also be shown that the semantic information is of the unique importance for the trinity. This is why we address a theory of semantic information without mentioning the syntactic and pragmatic information.

It is proved that there are no unknown perfect (Hamming-)error-correcting codes over finite fields.

Coding technology is used in several information processing tasks. In particular, when noise during transmission disturbs communications, coding technology is employed to protect the information. However, there are two types of coding technology: coding in classical information theory and coding in quantum information theory. Although the physical media used to transmit information ultimately obey quantum mechanics, we need to choose the type of coding depending on the kind of information device, classical or quantum, that is being used. In both branches of information theory, there are many elegant theoretical results under the ideal assumption that an infinitely large system is available. In a realistic situation, we need to account for finite size effects. The present paper reviews finite size effects in classical and quantum information theory with respect to various topics, including applied aspects.

Multiple sequence alignments (MSAs) of homologous proteins are useful tools to characterize compensatory mutations between non-conserved residue sites. The identification of these residue sites in MSAs is an important challenge for understanding the structural basis and molecular mechanism of protein functions. Despite the rich literature on compensatory mutations as well as sequence conservation analysis for the identification of those important residue sites, previous methods often do not take into account biochemical constraints of amino acids which are likely to be crucial for the detection of compensatory mutation signals. However, compensatory mutation signals in MSAs are often masked by noise. Thus, another challenging problem in bioinformatics is the separation of significant signals from the phylogenetic noise and unrelated pair signals. The goal of this thesis is to develop such methods that incorporate biochemical constraints like similarities or dissimilarities of amino acids in identifying compensatory mutations and deal with the noise. Hence, we develop different methods based on classical and quantum information theory and multiple testing procedures. Our first method is based on classical information theory. It mainly focuses on BLOSUM62-dissimilar amino acid pairs as a model of compensatory mutations and incorporates them in the prediction of functionally and/or structurally important sites using a doubly stochastic matrix. To complement this method, we develop our second method applying principles of quantum information theory. The new method differs from the first one by simultaneously modeling similar and dissimilar amino acid pair signals in the compensatory mutation analysis. Moreover, to separate method-based significant compensatory mutation signals from background noise, we develop an MSA-specific statistical model devised for multiple testing problems. By applying this model, we are capable of determining significant signals in MSAs as well as quantifying the error made in terms of the false discovery rate. To demonstrate the effectiveness of our methods, we evaluate those analyzing important sites of two human proteins, namely epidermal growth factor receptor (EGFR) and glucokinase (GCK). Our results suggest that the MSA-specific statistical model is able to separate significant compensatory mutation signals from the phylogenetic noise and unrelated pair signals. Only considering the dissimilarities of amino acids, the first method successfully deals with disease-associated important sites of both proteins. In contrast, simultaneously focusing on similar and dissimilar amino acid signals, the second method is more sensible to catalytic, allosteric and binding sites. The results further show that overlaps between both methods are quite low, indicating that considerably different sets of residue sites are detected by both methods as functionally and structurally important. As a result of this, we can say that our second method complements the first method when it comes to predicting important sites, rather than replacing it.

Existing theories of creek and pan formation in intertidal salt marshes emphasise intertidal micro‐topography and resultant variation in vegetal colonisation as the basic factors controlling their pattern on the mature salt marsh. Morphological studies on certain Scottish marshes suggest that collapse of parts of the mature marsh surface results from sub‐surface piping phenomena. The existing theories of creek and pan formation are reviewed and an entirely new theory of formation is advanced.

Information theory, as it is known today, resulted from the confluence of two very different roots that had their origins in the first half of the 20th century. On the one hand, information theory originated from electrical engineers such as Hartley, Nyquist, and Shannon [49, 86, 104], who worked on the analysis of systems and strategies to communicate messages from one location to another. On the other hand, mathematicians such as de Bruijn, Cramér, Fisher, Kullbach, and Rao were inventing ideas in probability and statistics that have direct relevance to the study of information transmission. In this chapter the "communications" aspect of information theory is emphasized, whereas in Chapter 3 the "probability and statistics" side was reviewed. In recent years, the theory of finite groups has been connected with equalities in information theory. Lie groups enter as symmetry operations associated with continuous physical models of information transmission such as the linear telegraph equation and nonlinear soliton equations. Lie groups also appear as a domain in which stochastic trajectories evolve in the analysis of noise in optical communication systems that transmit information over fiber optic cables. In addition, some of the basic concepts and definitions in the theory of communication have interesting properties that are enriched by merging them with concepts from group theory. Some of this recent work will be explored here.

This paper seeks to discuss why information theory is so important. What is information, knowledge is interaction of human mind and information, but there is a difference between information theory and knowledge theory. Look into information and particle theory and see how information must have its roots in particle theory. This leads to the concept of spatial dimensions, information density, complexity, particle density, can there be particle complexity, and re-looking at the double slit experiment and quantum tunneling. Information functions/ relations are discussed.

The ideas of information theory which underlie cybernetics and its information‐processing aspects still has a solid reputation to be both extremely abstract and highly mathematical. Furthermore, the tract has been accused of a slow development and not accentuating the difference between information and communication theories. In this paper an attempt is made to demonstrate that principles and concepts used in information theory can be both understood and used from a psychological basis ‐ without the use of complicated mathematics. The difference between information and communication theory is considered as well as some new and revolutionary concepts. The study shows that the influences of the investigated area have grown considerably with the contributions from information physics.

1. Introduction. Transdisciplinarity and Information Science and Philosophy The emergence of the concept of transdisciplinarity, and the field of transdisciplinary studies has come in response not only to the proliferation of disciplines and the need to manage their practical applications, but to the post-modernist crisis in the related philosophies of science and knowledge in general. Transdisciplinarity is not a new discipline but a philosophical movement which, through its non-standard logic of human experience and human intelligence, can provide a new approach to on-going problems and paradoxes of human thought, science and philosophy. In parallel to the development of transdisciplinarity, the last decade has also seen major developments not only in the information and communications technologies, but in the science and philosophy of information. As I will show, recent theories of information science and philosophy have a close relation to transdisciplinarity. 2. The Philosophy of Information as Metaphilosophy Starting in 1980 from philosophical considerations of the essence of information, Wu Kun, working at the Jiaotong University in Xi'an, China, developed a Philosophy of Information (PI) that included an informational ontology, epistemology and theory of social evolution. For Wu, information is a critical component of all disciplines, beyond the formal content specific to each. A small fraction of this work became available in English in 2010 in a monograph presented at an International Conference on the Foundations of Information Science in Xi'An ("The Basic Theory of the Philosophy of Information" (BTPI) [1]). In the conception of Wu Kun, the presence of information throughout existence converts the philosophy of information to a metaphilosophy. The comprehensive nature of such a metaphilosophy establishes a role of those involved in the social and ethical aspects of informational activities. The metaphilosophy of information requires attention to the informational aspects of complex processes as a methodological necessity, in a process that Wu calls Informational Thinking. Informational Thinking (IT), as conceived of by Wu, refers to a way of grasping and describing the essential characteristics and attributes of things by reference to the structure and dynamics of the information involved in their evolution, from their historical origins to future possibilities and probabilities. A summary of his views in English can be found in [2]. At the heart of Wu's theory is a necessarily alternative worldview that emphasizes its relational and process aspects. Information and informational processes, in the conception of Wu Kun, are transdisciplinary and can also be seen to evolve according to the principles of Logic in Reality (see below). This work was completely independent of the concomitant development of a Philosophy of Information by Luciano Floridi [3], working at the University of Hertfordshire in the U.K. The differences in the two approaches are philosophical: Floridi's theory is basically epistemological, seeing the operation of information from the perspective of the human observer-reasoner. The philosophy of information has thus transcended its origins in information and computation science and technology. We move from a quantitative, "technological" conception of information to what may fairly be called a transdisciplinary one. It is not only that the philosophy and metaphilosophy of information refer to the standard disciplines that makes them transdisciplinary but that they contain, like transdisciplinarity in general, what lies in, between and beyond the different conceptions of information – an attitude, a stance and an ethics. The 2nd International Conference on the Philosophy of Information is taking place as a major Stream within this Summit. 3. The Logic of Transdisciplinarity I consider that a more scientific description of the grounding of ethical human ethical behavior is not just an intellectual exercise but a moral obligation. Two related tools now available for this task are thus transdisciplinarity and informational science and philosophy or metaphilosophy as indicated above. A necessary component of both is the non-standard logic of transdisciplinarity originally proposed by Lupasco [4] and up-dated by me and made available to English-language readers as Logic in Reality (LIR) [5]. In the acceptation of Basarab Nicolescu [6], the three 'pillars' of transdisciplinarity are complexity, levels of reality and this logic of the included middle or third. In previous papers, I have also discussed in some detail my interpretation of the Lupasco system as a non-truth-functional, non-linguistic extension of logic to real systems. A key axiom defines the energetic logical relations between the opposing or contradictory elements of real processes. I emphasize that both approaches include the emergence of new states through the principle of dynamic opposition, the dialectic and interactive relation between the dual elements of all real processes. The difference, very briefly, is the following: Nicolescu looks 'upward' toward the transcendental aspects of existence, extending the Lupasco logic to cover the relations between epistemological Subjects and Objects, designated as Transdisciplinary, at higher levels of cognitive reality. LIR focuses on the explication of the evolution of complex real systems, their ontological subjects and objects, and the information processes directly associated with them. This point is critical for the discussion of transdisciplinarity in relation to information science. My view of transdisciplinarity and its relation to a logic is similar to the discussion by Roderick Lawrence in his paper "Transgression of Disciplinary Frontiers" [7]. In particular, he cites the statement by Thierry Ramadier that "the specificity of transdisciplinarity consists in simultaneously integrating two contradictory movements (emphasis mine) of disciplinary logic, that is, the fragmentation of knowledge and the relation between the "fragments", in order to do research into the connections possible between the (forms of) knowledge produced". These are the kinds of movements, including their connections to the fundamental physics of our world, which Logic in Reality can describe. 4. Applications in the Real World 4.1 The Global Sustainable Information Society The concept of Wolfgang Hofkirchner and his associates in Salzburg and (now) Vienna is that the study of the emerging theory of the information society is transdisciplinary. In particular the new field of research in the Information and Communication Technologies (ICTs) and Society is a transdiscipline, as proposed in 2007 [8]. The key aspects of a transdiscipline for Hofkirchner et al. are its scientific status and its potential societal function. As regards the scientific status of the field, a transdiscipline is not a mere combination of existing disciplines but a transgression of their traditional borders and their transformation into something new with its own identity. Its terminology should overarch the terminologies of the single disciplines it departs from. A transdiscipline therefore is expected to bridge several gaps: the gap between the two cultures of (natural) science and social and human sciences as well as the gap between specialists and generalists as well as the gap between applied research and basic research. It is the result of a process that departs from mono- or multidisciplinarity and transcends interdisciplinarity. In this view, it is the role of a transdisciplinary information science [8] to help in bringing about a Global Sustainable Information Society (GSIS) [9]. A GSIS can be defined in a normative way and technology (the ICTs) can be assessed according to how they facilitate society to achieve the GSIS. This is in sharp contrast to either undertaking research solely for reasons of curiosity or being instrumental to whatever is demanded by parts of society. In contrast to the ideology of value-free science, the normative criteria are laid down to which ICTs as well as society should be subject. Hofkirchner argues that transdisciplinary features must inhere to the newly established field of ICTs-and-Society research if it is to 1) be critical of current socio-economic developments; 2) aim for the establishment of a GSIS; 3) tackle the complex problems of society and technology; and 4) use social-scientific and technological, empirical and theoretical methods in a proper way. Logic in Reality (LIR) supports this transdisciplinary view as it involves integrative ICT assessment and design approaches that incorporate a normative view of technology and society. There is no place in LIR for value-free science; the practitioner is always involved logically with the material substrate of his science, whose dynamics and properties he partly shares. As clearly stated by Hofkirchner et al., a normative approach requires "doing justice" to what is normative and factual, actual and potential. The term "transdiscipline" should be adopted in discussions of transdisciplinarity where it brings out better the issues under discussion. Whether the use of the term conflicts with a definition of transdisciplinarity which is also supposed to be beyond all disciplines is for me a secondary question, perhaps best answered pragmatically by reference to transdisciplinary openness itself. 4.2 The Ethical Dimension and the Environment As discussed above, the link between informational philosophy and transdisciplinarity is the logic of and in reality (LIR), which is, also, the logic of transdisciplinarity. A basic tenet of this logic is a respect for the other, as stated by Nicolescu in his Manifesto [10]. The other in the broadest sense is not only female vs. male in a male-dominated society and racial and ethic minorities in general but human and non-human, that is, the total physical environment. In the LIR view of ethical behavior, the same metaphysical but also physical principle of dynamic opposition provides the basis for both 1) a generally applicable antagonistic psychological typology of responsible and irresponsible behavior; and 2) the origin of environmental responsibility and in fact moral responsibility in general. Morality in the generally accepted sense of responsibility toward others as well as oneself and the environment is thus logically and ontologically grounded, as are other universal aspects of human behavior and not dependent on transcendental assumptions that serve only to weaken its purport. Strategies to strengthen awareness of and positive response to environmental threats should thus emphasize common humanity and a common psychological structure across cultures as well as enlightened self-interest. This area is being currently addressed by Zong-Rong Li and his associates [11]. Li has suggested the term 'Informationalism' to capture the controlling function of informational existence in which information science and material science explain individual and social phenomena. This approach permits, among other things, a reformulation and interpretation of psychology and its history into a specifically informational psychology. 4.3 Toward a New Democratic System In the applications suggested by Wu Kun for his theory and philosophy of information, no specific comprehensive economic-political model is suggested, but he does call for a "new democratic system" that would permit maximization of the benefits from the new ICTs. A proper model would include an informational perspective for studying social phenomena, a social information theory based on his concept of the essence of information in a social evolutionary context. As shown in Wu [1], forms of human civilization can be differentiated according to their different ways of creating, processing, dissemination and development of information. Only human beings can create information. Human production and productivity are essentially only information production and information productivity, and models of the economy and market activity are informational models. The expanded social role of information should be accompanied by the development of networks for its dissemination resulting in the (slow) atrophy of centralized national and global hegemony. However, any theory or model of social change cannot ignore (see my interpretation of the Lupasco logic above) the inevitable fundamental embodiment of contrary, anti-social and anti-civilizational forces in the society. These will always make the struggle for the common good and implementation of human values a struggle indeed. 5. Summary and Conclusions In summary, one of the most important aspects of the concept of transdisciplinarity is its relation to the field of information and information science. The role and function of the Logic in Reality as the logic of transdisciplinarity is to support, philosophically and scientifically, the transdisciplinary approach or attitude toward current issues in philosophy and science and to provide new insights into the qualitative, ethical aspects of the informational evolution of science and society. I believe that a new way of looking at thought and the traditional disciplines can make a contribution to the establishment of an informational commons. Following Wu Kun, I have shown that the philosophy of information is a metaphilosophy that also makes possible a new conception of nature, understanding, society and values and actively promotes the development of human information society, and a more civilized and democratic social polity, economic and cultural new order. Wu's informational philosophy and LIR constitute part of a new transdisciplinary paradigm, in which information science has a central role in the transformation of society. References Wu, K. 2010. The Basic Theory of Philosophy of Information. Paper, 4th International Conference on the Foundations of Information Science, August, 2010, Beijing. Brenner, J. E. 2010. Wu Kun and the Metaphilosophy of Information. International Journal "Information Theories and Applications", 18(2), 103-128. Floridi L. 2010. The Philosophy of Information. Oxford University Press: Oxford, U.K. Lupasco S. 1987. Le principe d'antagonisme et la logique de l'énergie. Paris : Editions du Rocher.(Originally published in Paris: Éditions Hermann, 1951). Brenner, J. E. 2008. Logic in Reality. Dordrecht: Springer. Nicolescu B. 2011. Methodology of Transdisciplinarity-Levels of Reality, Logic of the Included Middle and Complexity. In Transdisciplinarity: Bridging Natural Science, Social Science, Humanities & Engineering, A. Ertas (ed.). www.theatlas.org/atlas-books.pdf , 22-45. Lawrence R.J. 2008 Transgresser les Frontières Disciplinaires. In Le Défi de l'Inter- et Transdisciplinarité, F. Darbellay, T. Paulsen (Eds.). Lausanne: Presses polytechniques et universitaires romandes, 223-238. Hofkirchner, W., Fuchs, C., Raffl, C., Schafranek, M., Sandoval, M. & Bichler, R. 2007. ICTs and Society: The Salzburg Approach. University of Salzburg Research Paper No. 3, December. Salzburg: ICT&S Center. Hofkirchner, W. 2013. Emergent Information; A Unified Theory of Information Framework. World Scientific Publishing Company: Singapore. Nicolescu B. 2002. Manifesto of Transdisciplinarity. Albany, NY: SUNY Press. Li, Z.-R. 2013. The Worldview and Methodology of Information Science. In International Conference on Philosophy of Information, October 18 -21, 2013 (ed. Z.-R. Li et al.), Xi'an, China.

Bounds for Truncation Error of the Sampling Expansion

Organizations and agencies release various data intended for analysis, training of artificial intelligence systems, and other research purposes. According to the adopted regulations in the field of personal data protection, public data must be anonymized and protected from various threats of personal data disclosure. Elimination of these threats is realized by reducing the accuracy of data during their preparation for the release. Loss of accuracy obviously leads to a decrease in the usefulness of data for analysis. The paper considers entropy metrics of utility and problems of their computability, as well as metrics of loss of utility of certain subsets of public data. Objective. To develop effective metrics for assessing the usefulness of a public dataset for analysis, taking into account the requirements of personal data protection. Research methods. Information security, Shannon's theory of information, Data Governance. Results. Metrics for assessing information loss and data usefulness for analysis based on the entropy metrics of Shannon's information theory are proposed. Procedures aimed at increasing the speed of calculations of the considered metrics are suggested. Conclusions. The procedures for building a secure public dataset are described. The application of entropy metrics of Shannon's information theory to assess information loss and data usefulness for analysis is considered. It has been shown that the calculation of these metrics is a complex computational task that is practically impossible for large databases. Procedures aimed at increasing the speed of calculating the considered metrics are proposed. In particular, the creation of a less accurate copy of the original data and the formation of a random sample from a large database to calculate the necessary statistics. The metrics for assessing the usefulness of certain subsets (clusters) of public data are considered in the article.

The motivation behind this paper is threefold, as follows. Firstly, it attempts to make a comprehensive exposition of the important role of elementary 'information theory', relevant to 'communication coding'. Secondly, the paper tries to illustrate comprehensively and convincingly how 'information theory' is a bridge between two different disciplines, namely 'communication and coding' and 'probability and statistics'. Thirdly, and most importantly, the paper leads to the conclusion that while we are trying to solve a problem in a particular area, e.g. 'electrical engineering', we ought to have a 'research group' with scientists from all relevant areas, for instance economics, probability and statistics, geology etc. The point is: by marrying expertise in seemingly diverse fields of knowledge we would vastly improve our chances of arriving at a gainful solution to the problem at hand, which would be practically optimal.