Binary Markov Sources Research Articles

Joint Decoding and Estimation of Spatio-Temporally Correlated Binary Sources

In the context of distributed joint source-channel coding, we conceive a joint decoding and estimation scheme for binary Markov sources exhibiting spatio-temporal correlation. The proposed scheme is designed based on the serial concatenation of a trellis coded modulation (TCM) scheme and a unity-rate code. The symbol-based maximum a posteriori algorithm employed for TCM decoding is modified in order to exploit the source correlation. The estimation of both the spatial and temporal correlation parameters is performed jointly with the iterative decoding, hence allowing the estimated parameters to be updated after each iteration. Our simulation results reveal that when both the spatial and temporal correlation parameters are unknown, the proposed joint decoding and estimation scheme approaches the performance to the ideal system relying on perfectly known correlation parameters, therefore, demonstrating the superiority of the proposed scheme.

Thermodynamics of the binary symmetric channel

We study a hidden Markov process which is the result of a transmission of the binary symmetric Markov source over the memoryless binary symmetric channel. This process has been studied extensively in Information Theory and is often used as a benchmark case for the so-called denoising algorithms. Exploiting the link between this process and the 1D Random Field Ising Model (RFIM), we are able to identify the Gibbs potential of the resulting Hidden Markov process. Moreover, we obtain a stronger bound on the memory decay rate. We conclude with a discussion on implications of our results for the development of denoising algorithms.

Exploitation of 2D binary source correlation using turbo block codes with fine-tuning

This article proposes a joint source-channel coding technique for two-dimensional (2D) binary Markov sources by using concatenated turbo block codes composed of two Bose, Chaudhuri, Hocquenghem (BCH) codes, of which output is followed by a rate-1 recursive systematic convolutional code. The source correlation of all rows and columns of the 2D source is well exploited by using a modified Bahl–Cocke–Jelinek–Raviv (BCJR) algorithm for the decoding of the BCH codes. Simulation results show that the proposed technique outperforms in terms of bit error rate the codes that exploits one-dimensional (1D) source correlation using the modified BCJR algorithm, and obviously the conventional system without source correlation exploitation. In order to further improve the performance, this article aims to make fine-tuning of the code parameters, given the source correlation property, that can achieve performance even closer to the theoretical limit than without the fine-tuning. Finally, results of image transmission simulations using two images, one having strong and the other weak 2D correlation, are presented to demonstrate the effectiveness of our proposed technique.

Distributed joint source-channel coding for relay systems exploiting source-relay correlation and source memory

In this article, we propose a distributed joint source-channel coding (DJSCC) technique that well exploits source-relay correlation as well as source memory structure simultaneously for transmitting binary Markov sources in a one-way relay system. The relay only extracts and forwards the source message to the destination, which implies imperfect decoding at the relay. The probability of errors occurring in the source-relay link can be regarded as source-relay correlation. The source-relay correlation can be estimated at the destination node and utilized in the iterative processing. In addition, the memory structure of the Markov source is also utilized at the destination. A modified version of the Bahl, Cocke, Jelinek, and Raviv (BCJR) algorithm is derived to exploit the memory structure of the Markov source. Extrinsic information transfer (EXIT) chart analysis is then performed to investigate convergence property of the proposed technique. Results of simulations conducted to evaluate the bit-error-rate (BER) performance and the EXIT chart analysis show that, by exploiting the source-relay correlation and source memory simultaneously, our proposed technique achieves significant performance gain, compared with the case where the correlation knowledge is not fully used.

Group Testing for Binary Markov Sources: Data-Driven Group Queries for Cooperative Sensor Networks

Group testing has been used in many applications to efficiently identify rare events in a large population. In this paper, the concept of group testing is generalized to applications with correlated source models to derive scheduling policies for sensors' adopting cooperative transmissions. The tenet of our work is that in a wireless sensor network it is advantageous to allocate the same channel dimensions to all sensor sources that have the same response to a sequence of queries or tests. That is, nodes that have the same data attributes should transmit as a cooperative super-source. Specifically, we consider the case where sensors' data are modeled spatially as a one-dimensional Markov chain. Two strategies are considered: the recursive algorithm and the tree-based algorithm. The recursive scheme allows us to illustrate the performance of group testing for finite populations while the tree-based algorithm is used to derive the achievable scaling performances of the class of group testing strategies as the number of sensors increases. We show that the total number of queries required to gather all sensors' data scales in the order of the joint entropy. A further generalization of this concept provides the basis of deriving efficient data-gathering algorithms for correlated sources.

Joint source-channel turbo coding for binary Markov sources

We investigate the construction of joint source-channel (JSC) turbo codes for the reliable communication of binary Markov sources over additive white Gaussian noise and Rayleigh fading channels. To exploit the source Markovian redundancy, the first constituent turbo decoder is designed according to a modified version of Berrou's original decoding algorithm that employs the Gaussian assumption for the extrinsic information. Due to interleaving, the second constituent decoder is unable to adopt the same decoding method; so its extrinsic information is appropriately adjusted via a weighted correction term. The turbo encoder is also optimized according to the Markovian source statistics and by allowing different or asymmetric constituent encoders. Simulation results demonstrate substantial gains over the original (unoptimized) Turbo codes, hence significantly reducing the performance gap to the Shannon limit. Finally, we show that our JSC coding system considerably outperforms tandem coding schemes for bit error rates smaller than 10/sup -4/, while enjoying a lower system complexity.

A performance analysis of a discrete-time priority queueing system with correlated arrivals

We present performance analysis of a discrete-time system with two priority queues and correlated arrivals. The arrival process to each priority queue consists of the superposition of the traffic generated by independent binary Markov sources with the arrivals to the two queues being independent of one another. The first step is the determination of the busy period of a queue with correlated arrivals. Next, the joint probability generating function of the two queue lengths is derived and the unknown boundary function is determined using the busy period of the high-priority queue. The next step is to determine closed form expressions for the mean and variance of the queue lengths. Finally, the results are extended to multiple priority queueing systems with multiple types of traffic sources. Numerical results that demonstrate the impact of the correlated arrival process in the system are presented.

Accurate bounds for the asymptotic constant in a statistical multiplexer with homogeneous generalized binary Markov sources

This paper considers the asymptotic tail distribution of the number of cells queued in a statistical multiplexer fed with homogeneous generalized binary Markov sources. As the asymptotic decay rate is easy to obtain, we focus our effort on bounding the asymptotic constant, which is dependent on the initial phase combination of the sources and is hard to compute even for a moderate number of sources. We derive upper and lower bounds for the asymptotic constant, taking the initial phase combination into account. Numerical experiments show the accuracy of these bounds. They also show that, while the asymptotic decay rates are the same, the variation of initial phase combination of the sources may significantly affect the asymptotic constants.

Accurate bounds for the asymptotic constant in a statistical multiplexer with homogeneous generalized binary Markov sources

This paper considers the asymptotic tail distribution of the number of cells queued in a statistical multiplexer fed with homogeneous generalized binary Markov sources. As the asymptotic decay rate is easy to obtain, we focus our effort on bounding the asymptotic constant, which is dependent on the initial phase combination of the sources and is hard to compute even for a moderate number of sources. We derive upper and lower bounds for the asymptotic constant, taking the initial phase combination into account. Numerical experiments show the accuracy of these bounds. They also show that, while the asymptotic decay rates are the same, the variation of initial phase combination of the sources may significantly affect the asymptotic constants.

A performance analysis of the ATM multiplexer with selective discarding of the cells during congestion

In this paper, we present a discrete-time queuing analysis of the ATM multiplexer fed by independent binary Markov sources with two priority classes of cells. Each source alternates between On, Off periods and each On period consists of a random number of sub-periods. Each sub-period generates geometrically distributed number of cells, first cell of a sub-period is assigned high-priority while remaining cells are assigned low-priority. During normal operation the system delivers both high- and low-priority traffic while during congestion delivers only high-priority cells to their destinations and discard low-priority cells. We determine the performance of the system before and after discarding of the low-priority cells. We determine probability generating function of the queue length in both cases under infinite buffer assumption. Then, the mean delay for infinite buffer and approximate cell loss probabilities for the finite buffer cases are presented as a function of the percentage of the high-priority traffic load.

Source-channel optimized trellis codes for bitonal image transmission over AWGN channels

We consider the design of trellis codes for transmission of binary images over additive white Gaussian noise (AWGN) channels. We first model the image as a binary asymmetric Markov source (BAMS) and then design source-channel optimized (SCO) trellis codes for the BAMS and AWGN channel. The SCO codes are shown to be superior to Ungerboeck's codes by approximately 1.1 dB (64-state code, 10(-5) bit error probability), We also show that a simple "mapping conversion" method can be used to improve the performance of Ungerboeck's codes by approximately 0.4 dB (also 64-state code and 10 (-5) bit error probability). We compare the proposed SCO system with a traditional tandem system consisting of a Huffman code, a convolutional code, an interleaver, and an Ungerboeck trellis code. The SCO system significantly outperforms the tandem system. Finally, using a facsimile image, we compare the image quality of an SCO code, an Ungerboeck code, and the tandem code, The SCO code yields the best reconstructed image quality at 4-5 dB channel SNR.

A transient discrete-time queueing analysis of the ATM multiplexer

In this paper, we present a transient discrete-time queueing analysis of the ATM multiplexer whose arrival process consists of the superposition of the traffic generated by independent binary Markov sources. The functional equation describing the ATM multiplexer has been transformed into a mathematically tractable form. This allows derivation of the transient probability generating functions of the queue length and the number of active sources in the system. Then, application of the final value theorem results in the corresponding steady-state probability generating functions, as well as packet delay. We also present closed form expressions for the transient and steady-state moments of the queue length. The pure transform approach used in the present analysis is an extension of the well-known classical method used in the transient analysis of single server queues with uncorrelated arrivals. As a result, the analysis is relatively easy to follow and gives an alternative solution of the ATM multiplexer that does not involve matrix operations. The matrix solutions usually assume that the probability generating matrix of the system has distinct eigenvalues, where the solution presented here does not have such restrictions. The paper presents significant new simple results on the transient analysis of the ATM multiplexer.

Analysis and design of trellis codes optimized for a binary symmetric Markov source with MAP detection

We consider the problem of transmitting a binary symmetric Markov source (BSMS), over the additive white Gaussian noise (AWGN) channel. The coding technique considered is trellis-coded modulation (TCM), where we utilize decoders which implement the maximum-likelihood (ML) and maximum a posteriori (MAP) criteria. Employing 8-PSK Ungerboeck codes on a BSMS with state transition probability 0.1, we first show that the MAP decoder realizes a 0.8-2.1-dB coding gain over the ML decoder. Motivated by these gains, we consider the design of trellis codes optimized for the BSMS/AWGN/MAP system. An approximate union bound is established for this system. Using this bound, we found codes which exhibit additional 0.4-1.1-dB gains over Ungerboeck codes. Finally, we compare the proposed TCM system with a tandem coding system. At normalized signal-to-noise ratio (SNR) of 10.8 dB and below, the proposed system significantly outperforms the tandem system.

Detection of binary Markov sources over channels with additive Markov noise

We consider maximum a posteriori (MAP) detection of a binary asymmetric Markov source transmitted over a binary Markov channel. The MAP detector observes a long (but finite) sequence of channel outputs and determines the most probable source sequence. In some cases, the MAP detector can be implemented by simple rules such as the "believe what you see" rule or the "guess zero (or one) regardless of what you see" rule. We provide necessary and sufficient conditions under which this is true. When these conditions are satisfied, the exact bit error probability of the sequence MAP detector can be determined. We examine in detail two special cases of the above source: (i) binary independent and identically distributed (i.i.d.) source and (ii) binary symmetric Markov source. In case (i), our simulations show that the performance of the MAP detector improves as the channel noise becomes more correlated. Furthermore, a comparison of the proposed system with a (substantially more complex) traditional tandem source-channel coding scheme portrays superior performance for the proposed scheme at relatively high channel bit error rates. In case (ii), analytical as well as simulation results show the existence of a "mismatch" between the source and the channel (the performance degrades as the channel noise becomes more correlated). This mismatch is reduced by the use of a simple rate-one convolutional encoder.

On the asymptotic analysis of statistical multiplexers with hyper-bursty arrivals

We consider multiplexers in discrete time fed by the superposition of “Ternary Markov Sources”. Such sources are the natural extension of the Binary Markov Sources (BMS) recently used to model bursty arrivals in a high speed environment. Unlike BMS, we allow sources to have arbitrary (large) variance in the duration of their OFF (silence) or ON (burst) periods. This paper focuses mainly on the impact of large variability either in the ON or OFF period on the performance. Following some asymptotic analysis, simple results on the tail behavior of the number of cells queued in the multiplexer are given. Our results indicate that ignoring the variability in the ON period may grossly underestimate the cell buildup in the multiplexer queue for all levels of the utilization. Furthermore, the impact of large variability of the OFF period depends very much on the utilization of the system. For a lightly-loaded multiplexer (utilization below a given threshold), the impact of large variability of the OFF period is minimal. However, for a heavy-loaded multiplexer (utilization above the threshold) the impact of the large variability in the OFF period is similar to that of the ON period.

On the performance of bursty and modulated sources subject to leaky bucket rate-based access control schemes

Provides an analysis of a rate-based access control scheme in high speed environments based on a buffered leaky bucket algorithm. The analysis is carried out in discrete time which is representative of an ATM environment. For the cell arrivals to the leaky bucket the authors consider a general discrete Markovian arrival process which models bursty and modulated sources. The key of the analysis is the introduction of the deficit junction that allows the reduction of the original problem to a more standard discrete time queueing system with the same arrival process. As an important special case, the detailed analysis of the binary Markov source throttled by such rate-based access control schemes is presented. Along with explicit recursions for computation of state probabilities and simple characterisation of the asymptotic behavior of the queue build up, some guidelines for the parameter selection of these schemes are provided. The results indicate that for sources with relatively large active periods, for an acceptable grade-of-service at the input queue, the token generation rate should be chosen to be close to the peak rate of the source, and increasing the bucket size of the leaky bucket does not improve substantially the performance at the input queue.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

Optimal detection of discrete Markov sources over discrete memoryless channels-applications to combined source-channel coding

The authors consider the problem of detecting a discrete Markov source which is transmitted across a discrete memoryless channel. Two maximum a posteriori (MAP) formulations are considered: (i) a sequence MAP detection in which the objective is to determine the most probable transmitted sequence given the observed sequence and (ii) an instantaneous MAP detection which is to determine the most probable transmitted symbol at time n given all the observations prior to and including time n. The solution to the first problem results in a "Viterbi-like" implementation of the MAP detector (with Large delay) while the latter problem results in a recursive implementation (with no delay). For the special case of the binary symmetric Markov source and binary symmetric channel, simulation results are presented and an analysis of these two systems yields explicit critical channel bit error rates above which the MAP detectors become useful. Applications of the MAP detection problem in a combined source-channel coding system are considered. Here, it is assumed that the source is highly correlated and that the source encoder (a vector quantizer (VQ)) fails to remove all of the source redundancy. The remaining redundancy at the output of the source encoder is referred to as the "residual" redundancy. It is shown, through simulation, that the residual redundancy can be used by the MAP detectors to combat channel errors. For small block sizes, the proposed system beats Farvardin and Vaishampayan's channel-optimized VQ by wide margins. Finally, it is shown that the instantaneous MAP detector can be combined with the VQ decoder to form an approximate minimum mean-squared error decoder.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

A simple method for calculating the rate distortion function of a source with an unknown parameter

The rate distortion function R( D) measures the minimum information rate of a source required to be transmitted at a fidelity level D. Although Blahut developed an elegant algorithm to calculate R( D) for discrete memoryless sources, computing R( D) for other types of sources is still very difficult. In this paper, we study the computation of R( D) for discrete sources with an unknown parameter which takes values in a continuous space. According to the well known ergodic decomposition theorem, a non-ergodic stationary source can be represented by a class of parameterized ergodic subsources with a known prior distribution. Based on this theory, a source matching approach and a simple algorithm is presented for computational purposes. The algorithm is shown to be convergent and efficient. In order to see the performance of this simple algorithm, we consider a special class of binary symmetric first-order Markov sources which has been previously studied. R( D) is computed over this class of sources and compared with the bound developed in previous work by Gray and Berger. The example shows that the algorithm is very efficient and produces results close to Gray and Berger's bound. Other examples further demonstrate the efficiency of the algorithm.

Some properties of sequential predictors for binary Markov sources

Universal predictions of the next outcome of a binary sequence drawn from a Markov source with unknown parameters is considered. For a given source, the predictability is defined as the least attainable expected fraction of prediction errors. A lower bound is derived on the maximum rate at which the predictability is asymptotically approached uniformly over all sources in the Markov class. This bound is achieved by a simple majority predictor. For Bernoulli sources, bounds on the large deviations performance are investigated. A lower bound is derived for the probability that the fraction of errors will exceed the predictability by a prescribed amount Delta >0. This bound is achieved by the same predictor if Delta is sufficiently small. >

An improved Ziv‐lempel coding scheme for universal source coding

AbstractThis paper proposes a universal information source code (improved ZL code) based on the Ziv‐Lempel (ZL) incremental parsing algorithm. It is known that a ZL code is asymptotically optimum for a stationary ergodic information source, although it does not utilize the source structure. Its convergence, however, is very slow and it cannot provide efficient compression for an input string of limited length. This paper considers the redundancy of ZL code, which is the reason for the slow convergence. In order to eliminate the redundancy a coding scheme is presented in which only the substring obtained by incremental parsing is used in the next stage. When the substrings of the incremental parsing are separately coded the improved ZL code obtained by the proposed scheme is the least redundant in the sense that the mean code length for the substrings is minimized. In the coding a data structure called modified ZL tree is introduced and its properties discussed together with two coding algorithms based on that structure. The coding is actually performed for the output string from a memory‐less binary source and binary Markov source. It is shown that sufficient compression is realized for a binary code string length of 103 to 104, indicating the possibility as a practical data compression algorithm.