Mathematical Theory Of Communication Research Articles

Application of a Translational Profiling Approach for the Comparative Analysis of CNS Cell Types

Comparative analysis can provide important insights into complex biological systems. As demonstrated in the accompanying paper, translating ribosome affinity purification (TRAP) permits comprehensive studies of translated mRNAs in genetically defined cell populations after physiological perturbations. To establish the generality of this approach, we present translational profiles for 24 CNS cell populations and identify known cell-specific and enriched transcripts for each population. We report thousands of cell-specific mRNAs that were not detected in whole-tissue microarray studies and provide examples that demonstrate the benefits deriving from comparative analysis. To provide a foundation for further biological and in silico studies, we provide a resource of 16 transgenic mouse lines, their corresponding anatomic characterization, and translational profiles for cell types from a variety of central nervous system structures. This resource will enable a wide spectrum of molecular and mechanistic studies of both well-known and previously uncharacterized neural cell populations.

On the use of information theory for the analysis of the relationship between neural and imaging signals

Functional magnetic resonance imaging (fMRI) is a widely used method for studying the neural basis of cognition and of sensory function. A potential problem in the interpretation of fMRI data is that fMRI measures neural activity only indirectly, as a local change of deoxyhemoglobin concentration due to the metabolic demands of neural function. To build correct sensory and cognitive maps in the human brain, it is thus crucial to understand whether fMRI and neural activity convey the same type of information about external correlates. While a substantial experimental effort has been devoted to the simultaneous recordings of hemodynamic and neural signals, so far, the development of analysis methods that elucidate how neural and hemodynamic signals represent sensory information has received less attention. In this article, we critically review why the analytical framework of information theory, the mathematical theory of communication, is ideally suited to this purpose. We review the principles of information theory and explain how they could be applied to the analysis of fMRI and neural signals. We show that a critical advantage of information theory over more traditional analysis paradigms commonly used in the fMRI literature is that it can elucidate, within a single framework, whether an empirically observed correlation between neural and fMRI signals reflects either a similar stimulus tuning or a common source of variability unrelated to the external stimuli. In addition, information theory determines the extent to which these shared sources of stimulus signal and of variability lead fMRI and neural signals to convey similar information about external correlates. We then illustrate the formalism by applying it to the analysis of the information carried by different bands of the local field potential. We conclude by discussing the current methodological challenges that need to be addressed to make the information-theoretic approach more robustly applicable to the simultaneous recordings of neural and imaging data.

Information, Noise and et al.

Information, Noise and et al.

Failure Notice

Failure Notice

Shannon – Boltzmann – Darwin: Redefining information (Part I)

A scientifically adequate theory of semiotic processes must ultimately be founded on a theory of that can unify the physical, biological, cognitive, and computational uses of the concept. Unfortunately, no such unification exists, and more importantly, the causal status of informational content remains ambiguous as a result. Lacking this grounding, semiotic theories have tended to be predominantly phenomenological taxonomies rather than dynamical explanations of the representational processes of natural systems. This paper argues that the problem of that prevents the development of a scientific semiotic theory is the necessity of analyzing it as a negative relationship: defined with respect to absence. This is cryptically implicit in concepts of design and function in biology, acknowledged in psychological and philosophical accounts of intentionality and content, and is explicitly formulated in the mathematical theory of communication (aka information theory). Beginning from the base established by Claude Shannon, which otherwise ignores issues of content, reference, and evaluation, this two part essay explores its relationship to two other higher-order theories that are also explicitly based on an analysis of absence: Boltzmann's theory of thermodynamic entropy (in Part I) and Darwin's theory of natural selection (in Part II). This comparison demonstrates that these theories are both formally homologous and hierarchically interdependent. Their synthesis into a general theory of entropy and provides the necessary grounding for theories of function and semiosis.

Theory-Jamming

Musical improvisation (or jam) suggested as a method of rigourous eclecticism required by the diffusion of communication theory.

Social complexity can drive vocal complexity: Group size and information in chickadee calls in Carolina chickadees

There is considerable variation in systems of vocal communication in animals. One hypothesis to explain this variation is that the complexity of the social group influences the groups vocal complexity. This social complexity hypothesis for communication is also central to recent arguments for human language origins, but experimental tests of the hypothesis are lacking. This study investigated whether group size, a fundamental component of social complexity, influenced the complexity of the chickadee call, a call system functioning in social organization of Carolina chickadees, Poecile carolinensis. This call system has been studied in chickadees and related species, and is structurally and functionally rich in its note composition. Complexity of calls was measured using uncertainty (diversity) related to note composition, based on Shannon’s mathematical theory of communication. In unmanipulated field settings, calls of individuals in larger groups had greater complexity (more information) than calls of individuals in smaller groups. In aviary settings manipulating group size, individuals in larger groups used calls with greater complexity than individuals in smaller groups. These results indicate that social complexity can influence communicative complexity in this species.

Senders, Receivers and Deceivers: How Liar Codes Put Noise Back on the Diagram of Transmission

Senders, Receivers and Deceivers: How Liar Codes Put Noise Back on the Diagram of Transmission

A semiotic analysis of the genetic information system

Terms loaded with informational connotations are often employed to refer to genes and their dynamics. Indeed, genes are usually perceived by biologists as basically ‘the carriers of hereditary information.’ Nevertheless, a number of researchers consider such talk as inadequate and ‘just metaphorical,’ thus expressing a skepticism about the use of the term ‘information’ and its derivatives in biology as a natural science. First, because the meaning of that term in biology is not as precise as it is, for instance, in the mathematical theory of communication. Second, because it seems to refer to a purported semantic property of genes without theoretically clarifying if any genuinely intrinsic semantics is involved. Biosemiotics, a field that attempts to analyze biological systems as semiotic systems, makes it possible to advance in the understanding of the concept of information in biology. From the perspective of Peircean biosemiotics, we develop here an account of genes as signs, including a detailed analysis of two fundamental processes in the genetic information system (transcription and protein synthesis) that have not been made so far in this field of research. Furthermore, we propose here an account of information based on Peircean semiotics and apply it to our analysis of transcription and protein synthesis.

Key to the management of the Western Himalayan Hillstreams in relation to

A key to the management of hillstreams in relation to Fish Species Richness (FSR) and Shannon and Weaver Diversity Index (H′) (Shannon and Weaver, 1949. The Mathematical Theory of Communications. Urbana, University of Illinois Press, IL, USA) using a multitude of factors such as altitude, water source, bed gradient, dominant channel materials and habitat type has been proposed on the basis of 10 study sites, belonging to three north Indian rivers namely; Ghaggar, Yamuna and Ganga. The selected streams are located between the altitudes ranging from 380 to 1524 m. It has been observed that high altitude streams having steep gradient, boulder dominated substrate with rapids as the main habitat type had lower FSR (3–4) and H′ (0.55–0.99) as compared to hillstreams located in the lower altitudes having gentle gradient, cobble dominated substrate with pools, riffles, runs, rapids and cascades as the main habitats and have high FSR (6–14) and H′ (1.67–2.35). Types – A, B and F hillstreams were encountered when the hillstreams are classified after Rosgen (Applied River Morphology, Wildland Hydrology, Co, USA).

Théorie de la cohésion par les renchaînements d’alliance dans les mariages et les réseaux sociaux

Ring cohesion, as a theory relevant to social cohesion, offers itself in the analysis of matrimonial relinking as an outgrowth of a structural approach: studies are, in the social sciences, the indirect outcome of modern developments in mathematics which have given increasing importance to the qualitative point of view in contradistinction to the quantitative point of view of traditional mathematics. It has become possible, therefore, in fields such as mathematical logic, set theory, group theory, and topology, to develop a rigorous approach to problems which do not admit of a metrical solution. The outstanding achievements in this connection Ð which offer themselves as springboards not yet utilized by social scientist - is to be found in J. von Neumann and O. Morgenstern, Theory of Games and Economic Behaviour; N. Wiener, Cybernetics; and C. Shannon and W. Weaver, The Mathematical Theory of Communication. [Levi-Strauss, Structural Anthropology, 1963, Chapter XV, Social Structure, section on Structure and Measure, p. 283].

Fields with finite information density

The existence of a natural ultraviolet cutoff at the Planck scale is widely expected. In a previous Letter, it has been proposed to model this cutoff as an information density bound by utilizing suitably generalized methods from the mathematical theory of communication. Here, we prove the mathematical conjectures that were made in this Letter.

Trace elements in marine birds, arctic hare and ringed seals breeding near Qikiqtarjuaq, Nunavut, Canada

offshore oil and gas exploration and production on the benthic communities of the Norwegian continental shelf. Marine Ecology Progress Series 122, 277–306. Pielou, E.C., 1966. Shannon’s formula as a measure of species diversity: its use and misuse. American Naturalist 100, 463–465. Rygg, B., 1985. Distribution of species along pollution-induced diversity gradients in benthic communities in Norwegian fjords. Marine Pollution Bulletin 16, 469–474. Shannon, E.C., Weaver, W., 1963. The mathematical theory of communication. University of Illinois Press, Urbana, USA. Shin, P.K.S., 1989. Natural disturbance of benthic infauna in the offshore waters of Hong Kong. Asian Marine Biology 6, 193– 207. Shin, P.K.S., 2000. The detection of ecological stress in a sub-tropical macrobenthic community in Hong Kong. Asian Marine Biology 17, 149–160. Shin, P.K.S., 2003. Changes in benthic infaunal communities in Tolo Harbour: will the trend continue? In: Morton, B. (Ed.), Perspectives on Marine Environment Change in Hong Kong and Southern China, 1977–2001. Proceedings of an International Workshop Reunion Conference, 21–26 October 2001, Hong Kong. Hong Kong University Press, Hong Kong, pp. 579–592. Shin, P.K.S., Fong, K.Y.S., 1999. Multiple discriminant analysis of marine sediment data. Marine Pollution Bulletin 39, 285–294. Shin, P.K.S., Thompson, G.B., 1982. Spatial distribution of the infaunal benthos of Hong Kong. Marine Ecology Progress Series 10, 37–47. Thompson, G.B., Shin, P.K.S., 1983. Sewage pollution and the infaunal benthos of Victoria Harbour, Hong Kong. Journal of Experimental Marine Biology and Ecology 67, 279–299. Wade, B.A., 1972. A description of a highly diverse soft-bottom community in Kingston Harbour, Jamaica. Marine Biology 13, 57–69. Wu, R.S.S., 1982. Periodic defaunation and recovery in subtropical epibenthic community, in relation to organic pollution. Journal of Experimental Biology and Ecology 64, 253–269. Wu, R.S.S., Cheung, R.Y.H., Shin, P.K.S., 1998. The ‘‘beneficial approach’’ in coastal management in Hong Kong: a compromise between rapid urban development and sustainable development. Ocean and Coastal Management 41, 89–102. Zar, J.H., 1996. Biostatistical Analysis, third ed. Prentice-Hall International, NJ, USA. Zhou, Q., He, M., Shao, H., Zhang, S., Huang, M., 1990. Distribution of lancelet and its relationship to bottom soil in Huangcuo waters of Xiamen. Journal of Oceanography of Taiwan Strait 9, 185– 189.

The face of the city is its information

What is it in the externally represented face of the city that makes it recognizable and imaginable? What makes some urban elements and artifacts more legible and better remembered than others? Or put more generally: What makes an object an external representation and what makes it better perceived and/or remembered? Our answer to all these questions is “the information they embody and convey”. Some elements, including those that compose the face of the city, are quantitatively and qualitatively more informative than others and are therefore more legible and better perceived and remembered, quantitatively, in terms of Shannon's theory of information (The Mathematical Theory of Communication, University of Illinois Press, Urbana, IL, 1949), and qualitatively in terms of Haken's (Information and Self-organization: a Macroscopic Approach to Complex Systems, Springer, Heidelberg, 1988/2000) notion of semantic information. In this paper, we elaborate on this point of view in three steps: In the first, we introduce Shannonian information and show how it can be used to define the amount of information externally represented in different urban elements. In the second, we show how this is related to processes of grouping and categorization that give meaning to the face of the city and thus form its semantic information. In the third, we discuss implications for cognition in general and for cognitive mapping in particular.

Knowledge and the mathematical properties of communication

This paper looks at some of the fundamental mathematical properties of communications. Knowledge is, we know, the property of a group of communicating agents yet little consideration is given to the features of these communications in the context of a group. Shannon & Weavers's landmark book, The Mathematical Theory of Communication, explored the properties of the information content within communication. This paper builds on this work by investigating the properties of communications within the ecology of Knowledge agents. The paper aims to feature some of the communication curiosities validated by mathematics and demonstrated in the physical world. These simple, but elegant, phenomena must be present in the more complex systems studied as KM systems and are required to build a substantial representation of knowledge in the internet age.

The Informational Turn in Philosophy

This paper traces the application of information theory to philosophical problems of mind and meaning from the earliest days of the creation of the mathematical theory of communication. The use of information theory to understand purposive behavior, learning, pattern recognition, and more marked the beginning of the naturalization of mind and meaning. From the inception of information theory, Wiener, Turing, and others began trying to show how to make a mind from informational and computational materials. Over the last 50 years, many philosophers saw different aspects of the naturalization of the mind, though few saw at once all of the pieces of the puzzle that we now know. Starting with Norbert Wiener himself, philosophers and information theorists used concepts from information theory to understand cognition. This paper provides a window on the historical sequence of contributions made to the overall project of naturalizing the mind by philosophers from Shannon, Wiener, and MacKay, to Dennett, Sayre, Dretske, Fodor, and Perry, among others. At some time between 1928 and 1948, American engineers and mathematicians began to talk about `Theory of Information' and `Information Theory,' understanding by these terms approximately and vaguely a theory for which Hartley's `amount of information' is a basic concept. I have been unable to find out when and by whom these names were first used. Hartley himself does not use them nor does he employ the term `Theory of Transmission of Information,' from which the two other shorter terms presumably were derived. It seems that Norbert Wiener and Claude Shannon were using them in the Mid-Forties. (Yehoshua Bar-Hillel, 1955)

The communication turn in the theory of social systems

AbstractThe theory of social systems has been contested as a part of sociology. Social order can be expected to emerge as an expectation that is communicated. A communication can be provided with meaning and meaning may evolve into codified meaning. The social system can thus be considered as developing in layers of communication among reflexive actors. At the interfaces between systems theory, communication theory and evolution theory puzzles emerge which can also be formulated as analytical and empirical questions. The mathematical theory of communication can be used for the clarification of the relations among these different perspectives, since a message is expected to contain information. Copyright © 2002 John Wiley & Sons, Ltd.

Entropy, coding and data compression

Rapid progress in all fields of scientific and technological activity has been the hallmark of the twentieth century. Though much of these was the result of the combined efforts of several scientists, some of the path-breaking developments were due to the original work of a few gifted ones. Claude Elwood Shannon (1916-2001) was one such in the field of communication science and technology. During the nineteen forti.es, he was working as a communications engineer in Bell Telephone Laboratories. The major problem at that time was reliable communication over radio, telephones, telegraphy, etc. The approach to these problems was that of A N Kolmogorov and N Wiener who advocated a model of a stochastic process of signals corrupted by disturbance, called noise, due to the medium used for transmission. The problem then is to recover the original signal from the corrupted one. Shannon made a radical departure from their approach by introducing the idea of encoding the signal before passing through the medium. In a single (two part) paper [1] entitled 'A Mathematical Theory of Communication' he not only provided a new set-up for the problem, but also proved a number of hasic theorems and gave a few practical methods to achievc what the theorems promised. He introduced the concept of entropy of a probability distribution [which already was in use in thermodynamics as a measure of disorder) and showed how this determines the minimum possible rate of reliable communication. He also named the new subject 'information theory since entropy is a measuT of the information content of a probabiiity distribut" The entropy idea was later taken to ergodic the 1958 by A N Kolmogorov to solve the outstand;

Dretske’s semantic information theory and meta‐theories in library and information science

This article presents the semantic information theory, formulated by the philosopher Fred I. Dretske, as a contribution to the discussion of metatheories and their practical implications in the field of library and information science. Dretske’s theory is described in Knowledge and the flow of information. It is founded on mathematical communication theory but developed and elaborated into a cognitive, functionalistic theory, is individually oriented, and deals with the content of information. The topics are: the information process from perception to cognition, and how concept formation takes place in terms of digitisation. Other important issues are the concepts of information and knowledge, truth and meaning. Semantic information theory can be used as a frame of reference in order to explain, clarify and refute concepts currently used in library and information science, and as the basis for critical reviews of elements of the cognitive viewpoint in IR, primarily the notion of “potential information”. The main contribution of the theory lies in a clarification of concepts, but there are still problems regarding the practical applications. More research is needed to combine philosophical discussions with the practice of information and library science.

A comparison of two information theory approaches to animal communication systems

Information theory has been used to analyze and explore animal communication systems using two different approaches. The first approach—which is the original application outlined by Shannon and Weaver (1949)—considers the temporal distribution (i.e., probabilities of occurrence) of the components of a communication system to quantify the diversity and complexity of a system’s (repertoires) organization. This quantification of the communication complexity is based on the information content, not just at the repertoire level, but also at the sequential (i.e., Markovian) organizational levels of the signals constituting the communication system. The second approach, which has been much more widely applied, with mixed results over several decades, considers a communication system’s ability to essentially ‘‘encode’’ signaling between individuals in order to measure how much information has been ‘‘transmitted.’’ We will outline and contrast these two approaches, pointing out that until the communication complexity of the entire signaling system is quantified, there is little hope of correctly interpreting information exchange between individuals using that system of communication [C. Shannon and W. Weaver, The Mathematical Theory of Communication (University of Illinois Press, 1949)].