Memoryless Stationary Source Research Articles

Compression-Based Compressed Sensing

Modern compression codes exploit signals’ complex structures to encode them very efficiently. On the other hand, compressed sensing algorithms recover “structured” signals from their under-determined set of linear measurements. Currently, there is a noticeable gap between the types of structures used in the area of compressed sensing and those employed by state-of-the-art compression codes. Recent results in the literature on deterministic signals aim at bridging this gap through devising compressed sensing decoders that employ compression codes. This paper focuses on structured stochastic processes and studies application of lossy compression codes to compressed sensing of such signals. The performance of the formerly proposed compressible signal pursuit (CSP) optimization is studied in this stochastic setting. It is proved that in the low-distortion regime, as the blocklength grows to infinity, the CSP optimization reliably and robustly recovers $n$ instances of a stationary process from its random linear measurements as long as $n$ is slightly more than $n$ times the rate-distortion dimension (RDD) of the source. It is also shown that under some regularity conditions, the RDD of a stationary process is equal to its information dimension. This connection establishes the optimality of CSP at least for memoryless stationary sources, which have known fundamental limits. Finally, it is shown that CSP combined by a family of universal variable-length fixed-distortion compression codes yields a family of universal compressed sensing recovery algorithms.

Achievable Error Exponents for the Private Fingerprinting Game

Fingerprinting systems in the presence of collusive attacks are analyzed as a game between a fingerprinter and a decoder on the one hand, and a coalition of two or more attackers on the other hand. The fingerprinter distributes, to different users, different fingerprinted copies of a host data (covertext), drawn from a memoryless stationary source, embedded with different fingerprints. The coalition members create a forgery of the data while aiming at erasing the fingerprints in order not to be detected. Their action is modeled by a multiple-access channel (MAC). We analyze the performance of two classes of decoders, associated with different kinds of error events. The decoder of the first class aims at detecting the entire coalition, whereas the second is satisfied with the detection of at least one member of the coalition. Both decoders have access to the original covertext data and observe the forgery in order to identify member(s) of the coalition. Motivated by a worst case approach, we assume that the coalition of attackers is informed of the hiding strategy taken by the fingerprinter and the decoder, while they are uninformed of the attacking scheme. Achievable single-letter expressions for the two kinds of error exponents are obtained. Single-letter lower bounds are also derived for the subclass of constant composition codes. These lower and the upper bounds coincide for the error exponent of the first class. Further, for the error of the first kind, a decoder that is optimal is introduced, and the worst case attack channel is characterized

On the Capacity Game of Private Fingerprinting Systems Under Collusion Attacks

The problem of fingerprinting in the presence of collusive attacks is considered. It is modeled as a game between a fingerprinter and a decoder on the one hand, and a coalition of two or more attackers on the other. The fingerprinter distributes, to different users, different fingerprinted copies of a host data (covertext ) embedded with different fingerprints. The coalition members create a forgery of the data while aiming at erasing the fingerprints in order not to be detected. Their action is modeled by a multiple-access channel (MAC). The decoder, who has access to the original covertext data, observes the forgery and decodes one of the messages in order to identify one of the members of the coalition. Motivated by a worst case approach, we assume that the coalition of attackers is informed of the hiding strategy taken by the fingerprinter and the decoder, while they are uninformed of the attacking scheme. A single-letter expression for the capacity is derived under the assumption that the host data is drawn from a memoryless stationary source and some mild assumptions on the operation of the encoder. It is shown that for a coalition consisting of L</spl infin/ members, the capacity scales with O(1/L), and whenever L grows with the length of the covertext, the capacity is essentially zero. Also, a lower bound on the error exponent is derived as a by-product of the achievability part, and asymptotically optimum strategies of the parties involved are characterized.

On the Capacity Game of Public Watermarking Systems

Watermarking codes are analyzed as a game between two players: an information hider and a decoder, on the one hand, and an attacker on the other hand. It is assumed that the covertext (the original data within which the message is hidden) is drawn from a memoryless stationary source and its realization is available at the information hider only. The information hider is allowed to cause some tolerable level of distortion to the covertext, and the resulting distorted data can suffer some additional amount of distortion caused by an attacker who aims at erasing the message. Motivated by a worst case approach, we assume that the attacker is informed of the hiding strategy taken by the information hider and the decoder, while they are uninformed of the attacking scheme. The capacity is expressed as the limit of a sequence of single-letter expressions under the assumption that the encoder uses constant composition codes.

On the error exponent and capacity games of private watermarking systems

Watermarking systems are analyzed as a game between an information hider, a decoder, and an attacker. The information hider is allowed to cause some tolerable level of distortion to the original data within which the message is hidden, and the resulting distorted data can suffer some additional amount of distortion caused by an attacker who aims at erasing the message. Two games are investigated: the error exponent game and the coding capacity game. Motivated by a worst case approach, we assume that the attacker is informed of the hiding strategy taken by the information hider and the decoder, which are uninformed of the attacking scheme. This approach leads to the maximin error exponent and maximin coding capacity as objective functions. It is assumed that the host data is drawn from a finite-alphabet memoryless stationary source, and its realization (side information) is available at the encoder and the decoder. A single-letter expression for the maximin error exponent is found under large deviations distortion constraints. Moreover, we find an asymptotically optimal random coding distribution, a universal decoder, and a worst case attack channel. It is proved that there is a saddle point in the asymptotic exponent and that the minimax and the maximin error exponents are equal. Finally, a single letter expression for the coding capacity, i.e., the maximin reliable information rate, is found.

Equivocations for transposition ciphers

In a transposition cipher the message sequence is subdivided into blocks of equal length and each block is shuffled according to a chosen key. To evaluate the performance of a transposition cipher we take a Shannon-theoretic standpoint and compute relevant equivocations. It is wellknown that transposition ciphers are strongly ideal for memoryless stationary sources (key and cryptogram are independent). We consider also message equivocation and key-appearance equivocation and do not assume that the length of the intercepted cryptogram is a multiple of the block-length. The index of coincidence of the source statistics is shown to be a relevant parameter, at least asymptotically.

The source coding theorem revisited: A combinatorial approach

A combinatorial approach is proposed for proving the classical source coding theorems for a finite memoryless stationary source (giving achievable rates and the error probability exponent). This approach provides a sound heuristic justification for the widespread appearence of entropy and divergence (Kullback's discrimination) in source coding. The results are based on the notion of composition class -- a set made up of all the distinct source sequences of a given length which are permutations of one another. The asymptotic growth rate of any composition class is precisely an entropy. For a finite memoryless constant source all members of a composition class have equal probability; the probability of any given class therefore is equal to the number of sequences in the class times the probability of an individual sequence in the class. The number of different composition classes is algebraic in block length, whereas the probability of a composition class is exponential, and the probability exponent is a divergence. Thus if a codeword is assigned to all sequences whose composition classes have rate less than some rate R , the probability of error is asymptotically the probability of the must probable composition class of rate greater than R . This is expressed in terms of a divergence. No use is made either of the law of large numbers or of Chebyshev's inequality.