Shannon Code Research Articles

Polynomial Representation of Binary Trees of Entropy Binary Codes

An important component of streaming large amounts of information are algorithms for compressing information flow. Which in turn are divided into lossless compression algorithms (entropic) - Shannon, Huffman, arithmetic coding, conditional compression - LZW, and otherinformation cone injections and lossy compression algorithms - such as mp3, jpeg and others.
 It is important to follow a formal strategy when building a lossy compression algorithm. It can be formulated as follows. After describing the set of objects that are atomic elements of exchange in the information flow, it is necessary to build an abstract scheme of this description, which will determine the boundary for abstract sections of this scheme, which begins the allowable losses.
 Approaches to the detection of an abstract scheme that generates compression algorithms with allowable losses can be obtained from the context of the subject area. For example, an audio stream compression algorithm can divide a signal into simple harmonics and leave among them those that are within a certain range of perception. Thus, the output signal is a certain abstraction of the input, which contains important information in accordance with the context of auditory perception of the audio stream and is represented by less information. A similar approach is used in the mp3 format, which is a compressed representation.
 Unlike lossy compression algorithms, entropic compression algorithms do not require contextanalysis, but can be built according to the frequency picture. Among the known algorithms for constructing such codes are the Shannon-Fano algorithm, the Huffman algorithm and arithmetic coding.
 Finding the information entropy for a given Shannon code is a trivial task. The inverse problem, namely finding the appropriate Shannon codes that have a predetermined entropy and with probabilities that are negative integer powers of two, is quite complex. It can be solved by direct search, but a significant disadvantage of this approach is its computational complexity. This article offers an alternative technique for finding such codes.

Attacks on the Random Gamming Code

In 1917, Hilbert Vernam patented a top-secret encryption scheme, which at first was called a one-time notepad and later a Vernam cipher. At the time that Vernam proposed this scheme, there was no evidence that it was completely secret, since, in fact, at that time yet there was no idea what the perfect secret of the cipher was. However, about 25 years later, Claude Shannon introduced the definition of perfect secrecy (perfect cipher) and demonstrated that the random gamming cipher reaches this level of security. Cryptographers believe that there are no effective attacks for attacks of random gamming. In particular, there are no effective attacks for the Vernam cipher.Objective: to justify the fallacy of this proposition to build effective attacks.Methods: analysis of the relationship between the cipher key and the received encrypted text.Results: an attack on the plaintext of a random gamming cipher based on a given encrypted text was developed. In addition, there was a suggestion for another attack on the plaintext contents based on the encrypted text. For all attacks, parameters of their complexity are calculated. These results are new. Previously, an attack on the random gamma code was unavailable. The results disprove the opinion that there are no attacks on this cipher.Practical relevance: firstly, it has become possible to carry out attacks on the random gamming code. Secondly, when using this cipher, it is necessary to strictly limit the length of the message.Discussion: the idea that there is an effective attack on a random gamming cipher arose in 2002, due to the possibility of introducing a similar concept, in which in a definition of the perfect cipher the plaintext is changed for a key. The first idea in creating attacks is that when the key is long its elements are repeated. The second idea is that attacks on two plaintexts are encrypted with one key. And the main idea was that it is necessary to improve the mathematical model of the Shannon code. Therein, when interpreting the concept of the perfect cipher, we should talk about the cipher model perfection.The publication place: in the Yandex search engine a query "Perfect ciphers" resulted in 22 million links, on a query "schemes perfectly secret" there were 43 million links. Yandex on the query "random gamming code" gave 13 million results.

CHARACTERIZATION OF CODES OF IDEALS OF THE POLYNOMIAL RING F30 2 [x] mod ô€€€ x30 ô€€€ 1 FOR ERROR CONTROL IN COMPUTER APPLICATONS

The study of ideals in algebraic number system has contributed immensely in preserving the notion of unique factorization in rings of algebraic integers and in proving Fermat's last Theorem. Recent research has revealed that ideals in Noethe-rian rings are closed in polynomial addition and multiplication.This property has been used to characterize the polynomial ring Fn 2 [x] mod (xn 1) for error control. In this research we generate ideals of the polynomial ring using GAP software and characterize the polycodewords using Shannon's Code region and Manin's bound.

Average Redundancy of the Shannon Code for Markov Sources

It is known that for memoryless sources, the average and maximal redundancy of fixed-to-variable length codes, such as the Shannon and Huffman codes, exhibit two modes of behavior for long blocks. It either converges to a limit or it has an oscillatory pattern, depending on the irrationality or rationality, respectively, of certain parameters that depend on the source. In this paper, we extend these findings, concerning the Shannon code, to the case of a Markov source. We provide a precise characterization of the convergent versus oscillatory behavior of the Shannon code redundancy for a class of irreducible, periodic, and aperiodic, Markov sources. These findings are obtained by analytic methods, such as Fourier/Fejér series analysis and spectral analysis of matrices.

How Suboptimal Is the Shannon Code?

In order to determine how suboptimal the Shannon code is, one should compare its performance with that of the optimal code, i.e., the corresponding Huffman code, in some sense. It is well known that in the worst case the redundancy of both the Shannon and Huffman codes can be arbitrarily close to 1. Beyond this worst case viewpoint, very little is known. In this paper, we compare the performance of these codes from an average point of view. The redundancy is considered as a random variable on the set of all sources with n symbols and its average is evaluated. It is shown that the average redundancy of the Shannon code is very close to 0.5 bits, whereas the average redundancy of the Huffman code is less than n-1(1+ln n)+0.086 bits . It is also proven that the variance of the redundancy of the Shannon code tends to zero as n increases. Therefore, for sources with alphabet size n, the redundancy of the Shannon code is approximately 0.5 bits with probability approaching 1 as n→ ∞.

Relations Between Redundancy Patterns of the Shannon Code and Wave Diffraction Patterns of Partially Disordered Media

The average redundancy of the Shannon code, <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Rn</i> , as a function of the block length <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">n</i> , is known to exhibit two very different types of behavior, depending on the rationality or irrationality of certain parameters of the source: It either converges to 1/2 as <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">n</i> grows without bound, or it may have a nonvanishing, oscillatory, (quasi-) periodic pattern around the value 1/2 for all large <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">n</i> . In this paper, we make an attempt to shed some insight into this erratic behavior of <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Rn</i> , by drawing an analogy with the realm of physics of wave propagation, in particular, the elementary theory of scattering and diffraction. It turns out that there are two types of behavior of wave diffraction patterns formed by crystals, which are correspondingly analogous to the two types of patterns of <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Rn</i> . When the crystal is perfect, the diffraction intensity spectrum exhibits very sharp peaks, a.k.a. Bragg peaks, at wavelengths of full constructive interference. These wavelengths correspond to the frequencies of the harmonic waves of the oscillatory mode of <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Rn</i> . On the other hand, when the crystal is imperfect and there is a considerable degree of disorder in its structure, the Bragg peaks disappear, and the behavior of this mode is analogous to the one where <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Rn</i> is convergent.

A Simple Recursive Shannon Code

The Shannon code is a very simple suboptimal scheme for computing prefix-free codelengths for a given memoryless source. A recursive version of the well-known Shannon code (RSh) is investigated in this paper. It has a redundancy which never exceeds that of the Shannon code. Unlike the Huffman code, the RSh code and its variations can be directly applied to sources with an infinitely countable alphabet. The average redundancy, taken over the set of all n-tuple distributions, is considered as a criterion for code comparisons. For n>;40, the average redundancy is approximately 0.14 bits for the RSh code, compared to approximately 0.5 and 0.03 bits for the Shannon and Huffman codes, respectively. For large n, a constrained version of the RSh code, called CRSh, has an average redundancy of only 0.07 bits for unsorted symbol probabilities. If the symbol probabilities are sorted in descending order, then a very simple modification of the RSh algorithm results in a code which is near-optimal from the average redundancy point of view.

The connection between information theory and object search in networks

This paper aims to establish a connection between information theory and object search in networks. We make two contributions:1) Using information theoretical arguments, we establish fundamental lower bounds on lookup table sizes and search step numbers. These bounds generalize those derived previously and can deal with non-uniform lookup distributions and object popularities. 2) Using the analogy between search sequences and data compression and coding, we propose a distributed implementation of Shannon code that can reduce the expected length of search sequences to an arbitrarily small value at the cost of at most doubling the table sizes.

Redundancy of Universal Coding, Kolmogorov Complexity, and Hausdorff Dimension

We study asymptotic code lengths of universal codes for parametric models. We show a universal code whose code length is asymptotically less than or equal to that of the minimum description length (MDL) code. Especially when some of the parameters of a source are not random reals, the coefficient of the logarithm in the formula of our universal code is less than that of the MDL code. We describe the redundancy in terms of Kolmogorov complexity and Hausdorff dimension. We show that our universal code is asymptotically optimal in the sense that the coefficient of the logarithm in the formula of the code length is minimal. Our universal code can be considered to be a natural extension of the Shannon code and the MDL code.

Precise Minimax Redundancy and Regret

Recent years have seen a resurgence of interest in redundancy of lossless coding. The redundancy (regret) of universal fixed-to-variable length coding for a class of sources determines by how much the actual code length exceeds the optimal (ideal over the class) code length. In a minimax scenario one finds the best code for the worst source either in the worst case (called also maximal minimax) or on average. We first study the worst case minimax redundancy over a class of stationary ergodic sources and replace Shtarkov's bound by an exact formula. Among others, we prove that a generalized Shannon code minimizes the worst case redundancy, derive asymptotically its redundancy, and establish some general properties. This allows us to obtain precise redundancy for memoryless, Markov, and renewal sources. For example, we present the exact constant of the redundancy for memoryless and Markov sources by showing that the integer nature of coding contributes log(logm/(m-1))/logm+o(1) where m is the size of the alphabet. Then we deal with the average minimax redundancy and regret. Our approach here is orthogonal to most recent research in this area since we aspire to show that asymptotically the average minimax redundancy is equivalent to the worst case minimax redundancy for some classes of sources. After formulating some general bounds relating these two redundancies, we prove our assertion for memoryless and Markov sources. Nevertheless, we provide evidence that maximal redundancy of renewal processes does not have the same leading term as the average minimax redundancy (however, our general results show that maximal and average regrets are asymptotically equivalent).

Classification of binary vectors by using ΔSC distance to minimize stochastic complexity

Stochastic complexity (SC) has been employed as a cost function for solving binary clustering problem using Shannon code length (CL distance) as the distance function. The CL distance, however, is defined for a given static clustering only, and it does not take into account of the changes in the class distribution during the clustering process. We propose a new ΔSC distance function, which is derived directly from the difference of the cost function value before and after the classification. The effect of the new distance function is demonstrated by implementing it with two clustering algorithms.

Source coding, large deviations, and approximate pattern matching

We present a development of parts of rate-distortion theory and pattern-matching algorithms for lossy data compression, centered around a lossy version of the asymptotic equipartition property (AEP). This treatment closely parallels the corresponding development in lossless compression, a point of view that was advanced in an important paper of Wyner and Ziv in 1989. In the lossless case, we review how the AEP underlies the analysis of the Lempel-Ziv algorithm by viewing it as a random code and reducing it to the idealized Shannon code. This also provides information about the redundancy of the Lempel-Ziv algorithm and about the asymptotic behavior of several relevant quantities. In the lossy case, we give various versions of the statement of the generalized AEP and we outline the general methodology of its proof via large deviations. Its relationship with Barron (1985) and Orey's (1985, 1986) generalized AEP is also discussed. The lossy AEP is applied to (i) prove strengthened versions, of Shannon's(1948, 1974) direct source-coding theorem and universal coding theorems; (ii) characterize the performance of mismatched codebooks in lossy data compression; ( iii) analyze the performance of pattern-matching algorithms for lossy compression (including Lempel-Ziv schemes); and (iv) determine the first-order asymptotic of waiting times between stationary processes. A refinement to the lossy AEP is then presented, and it is used to (i) prove second-order (direct and converse) lossy source-coding theorems, including universal coding theorems; (ii) characterize which sources are quantitatively easier to compress; (iii) determine the second-order asymptotic of waiting times between stationary processes; and (iv) determine the precise asymptotic behavior of longest match-lengths between stationary processes. Finally, we discuss extensions of the above framework and results to random fields.

Maximal codeword lengths in Huffman codes

In this paper, we consider the following question about Huffman coding, which is an important technique for compressing data from a discrete source. If p is the smallest source probability, how long, in terms of p, can the longest Huffman codeword be? We show that if p is in the range 0 < p ≤ 1 2 , and if K is the unique index such that 1 F K+3 < p ≤ 1 F K+2 , where F K denotes the K th Fibonacci number, then the longest Huffman codeword for a source whose least probability is p is at most K, and no better bound is possible. Asymptotically, this implies the surprising fact that for small values of p, a Huffman code's longest codeword can be as much as 44% larger than that of the corresponding Shannon code.

Lançando anzóis: uma análise cognitiva de processos mentais em tradução

TIle search for a general concept which would explain concisely the nature of the translation process has been an ongoing theme within the field of Translation TIleory. Lately, this search has pondered on the nature of translation both as a product and a process and produced several attempts towards such a concept. However, ali of these attempts have remained, at their best, at a c1assificatory leveI. Trying to fil! this conceptual gap, this article criticizes the applicability of concepts such as equivalence and adequacy within the field of Translation Theory and, then, drawing on Sperber & Wilson (986), it offers a complementary view, based on Relevance TIleory, which amalgamates both Shannon and Weaver's (1949) Code Model and Grice's (957) lnferential Model. The theoretical discussion is illustrated with empirical material collected by means of thinking-aloud protocols from German/ Portuguese translations.

The Renyi redundancy of generalized Huffman codes

Huffman's algorithm gives optimal codes, as measured by average codeword length, and the redundancy can be measured as the difference between the average codeword length and Shannon's entropy. If the objective function is replaced by an exponentially weighted average, then a simple modification of Huffman's algorithm gives optimal codes. The redundancy can now be measured as the difference between this new average and A. Renyi's (1961) generalization of Shannon's entropy. By decreasing some of the codeword lengths in a Shannon code, the upper bound on the redundancy given in the standard proof of the noiseless source coding theorem is improved. The lower bound is improved by randomizing between codeword lengths, allowing linear programming techniques to be used on an integer programming problem. These bounds are shown to be asymptotically equal. The results are generalized to the Renyi case and are related to R.G. Gallager's (1978) bound on the redundancy of Huffman codes. >