Symbol Probabilities Research Articles

Embedding Compression in Chaos-Based Cryptography

An algorithm for embedding compression in the Baptista-type chaotic cryptosystem is proposed. The lookup table used for encryption is determined adaptively by the probability of occurrence of plaintext symbols. As a result, more probable symbols will have a higher chance to be visited by the chaotic search trajectory. The required number of iterations is small and can be represented by a short code. The compression capability is thus achieved. Simulation results show that the compression performance on standard test files is satisfactory while the security is not compromised. Our scheme also guarantees that the ciphertext is not longer than the plaintext.

Iterative Joint Channel Estimation and Multiuser Detection for DS-CDMA in Frequency-Selective Fading Channels

An iterative joint channel estimation, symbol detection, phase recovery and interference cancellation structure is proposed for asynchronous code-division multiple-access systems over frequency-selective fading channels. Based on the expectation maximization (EM) algorithm, a recursive channel estimator is developed for blind channel tracking, using a novel stochastic signal processing technique. To perform symbol detection given the phase ambiguities of the resultant EM channel estimates, a noncoherent scheme is developed to compute the a posteriori probabilities (APPs) of data symbols. Moreover, by incorporating the APPs into the proposed recursive channel estimator, phase ambiguity due to the EM channel estimation can be resolved, which enables soft multiple access interference cancellation for multiuser detection. Based on these new signal processing schemes, an iterative structure is proposed for joint channel estimation and multiuser detection over fast fading channels.

Entropy of Patterns of i.i.d. Sequences—Part I: General Bounds

Tight bounds on the block entropy of patterns of sequences generated by independent and identically distributed (i.i.d.) sources are derived. A pattern of a sequence is a sequence of integer indices with each index representing the order of first occurrence of the respective symbol in the original sequence. Since a pattern is the result of data processing on the original sequence, its entropy cannot be larger. Bounds derived here describe the pattern entropy as function of the original i.i.d. source entropy, the alphabet size, the symbol probabilities, and their arrangement in the probability space. Matching upper and lower bounds derived provide a useful tool for very accurate approximations of pattern block entropies for various distributions, and for assessing the decrease of the pattern entropy from that of the original i.i.d. sequence.

A Divergence Minimization Approach to Joint Multiuser Decoding for Coded CDMA

In this paper, a theoretical framework of divergence minimization (DM) is applied to derive iterative receiver algorithms for coded CDMA systems. The DM receiver obtained performs joint channel estimation, multiuser decoding, and noise- covariance estimation. While its structure is similar to that of many ad-hoc receivers in the literature, the DM receiver is the result of applying a formal framework for optimization without further simplifications, namely the DM approach with a factorizable auxiliary model distribution. The well-known expectation- maximization (EM) algorithm and space-alternating generalized expectation-maximization (SAGE) algorithm are special cases of degenerate model distributions within the DM framework. Furthermore, many ad-hoc receiver structures from literature are shown to represent approximations of the proposed DM receiver. The DM receiver has four interesting properties that all result directly from applying the formal framework: (i) The covariances of all estimates involved are taken into account, (ii) The residual interference after interference cancellation is handled by the noise-covariance estimation as opposed to by LMMSE filters in other receivers, (iii) Posterior probabilities of the code symbols are employed rather than extrinsic probabilities, (iv) The iterative receiver is guaranteed to converge in divergence. The theoretical insights are illustrated by simulation results.

Low complexity soft-input soft-output block decision feedback equalization

A low complexity soft-input soft-output (SISO) block decision feedback equalizer (BDFE) is presented for turbo equalization. The proposed method employs a sub-optimum sequence-based detection, where the soft-output of the equalizer is calculated by evaluating an approximation of the sequence-based a posteriori probability (APP) of the data symbol. The sequence-based APP approximation is enabled by the adoption of both soft a priori information and soft decision feedback, and it leads to better performance and faster convergence compared to symbol-based detection methods as used by most other low complexity equalizers. The performance and convergence property of the proposed algorithm is analyzed by using extrinsic information transfer (EXIT) chart. Both analytical and simulation results show that the new equalizer can achieve a performance similar to that of trellis-based equalization algorithms, with a complexity similar to linear SISO minimum mean square error equalizers.

Computation of distributions and their moments in the trellis

Consider a function whose set of vector arguments with known distribution is described by a trellis. For a certain class of functions, the distribution of the function values can be calculated in the trellis. The forward/backward recursion known from the BCJR algorithm [2] is generalized to compute the moments of these distributions. In analogy to the symbol probabilities, by introducing a constraint at a certain depth in the trellis we obtain symbol distributions and symbol moments, respectively. These moments are required for an efficient implementation of the discriminated belief propagation algorithm in [8], and can furthermore be utilized to compute conditional entropies in the trellis.The moment computation algorithm has the same asymptotic complexity as the BCJR algorithm. It is applicable to any commutative semi-ring, thus actually providing a generalization of the Viterbi algorithm [10].

Soft-Output Decoding of Rotationally Invariant Codes Over Channels With Phase Noise

<para xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> We consider rotationally invariant (RI) trellis-coded modulations transmitted over channels affected by phase noise. To describe the main ideas of this paper, we first concentrate, as a case study, on the simplest RI scheme, namely the differentially encoded <emphasis emphasistype="italic">M</emphasis>-ary phase-shift keying signal. For this problem, we use the framework based on factor graphs and the sum–product algorithm to derive the exact maximum <emphasis emphasistype="italic">a posteriori</emphasis> symbol detection algorithm. By analyzing its properties, we demonstrate that it can be implemented by a forward–backward estimator of the phase probability density function, followed by a symbol-by-symbol completion to produce the <emphasis emphasistype="italic">a posteriori </emphasis> probabilities of the information symbols. To practically implement the forward–backward phase estimator, we propose a couple of schemes with different complexity. The resulting algorithms exhibit an excellent performance, and in one case, only a limited complexity increases with respect to the algorithm that perfectly knows the channel phase. The properties of the optimal decoder and the proposed practical decoding schemes are then extended to the case of a generic RI code. The proposed soft-output algorithms can also be used in iterative decoding schemes for concatenated codes employing RI inner components. Among them, in the numerical results, we consider repeat-accumulate codes and other serially concatenated schemes recently proposed in the technical literature. </para>

Uncoded optimal binary communication for sources with memory using the Burrows Wheeler Transform

Given an AWGN channel, we look at the problem of designing an optimal binary uncoded communication system for transmitting blocks of binary symbols generated by a stationary source with memory modelled by a Markov chain (MC) or a hidden Markov model (HMM). The goal is to minimize the average SNR required for a given block error rate. The particular case where the binary source is memoryless with nonuniform symbol probabilities has been studied by Korn et al. [Optimal binary communication with nonequal probabilities. IEEE Trans Commun 2003;51:1435–8] [1] by optimally allocating the energies of the transmitted signals. In this paper we generalize the previous work to include the important case of sources with memory. The proposed system integrates the block sorting Burrows Wheeler Transform (BWT, [Burrows M, Wheeler D. A block sorting lossless data compression algorithm. Research report 124. Digital Systems Center, 1994]) [2] with an optimal energy allocation scheme based on the first order probabilities of the transformed symbols. Analytical expressions are derived for the energy gain obtained with the proposed system when compared either with the optimal blockwise MAP receiver or with a standard source coded system consisting of an optimal source encoder followed by an optimal uncoded binary communication system, i.e. by a symbol-by-symbol MAP detector.

A multimedia framework for effective language training

We present a novel framework for the multimodal display of words using topological, appearance, and auditory representations. The methods are designed for effective language training and serve as a learning aid for individuals with dyslexia. Our topological code decomposes the word into its syllables and displays it graphically as a tree structure. The appearance code assigns color attributes and shape primitives to each letter and takes into account conditional symbol probabilities, code ambiguities, and phonologically confusable letter combinations. An additional auditory code assigns midi events to each symbol and thus generates a melody for each input string. The entire framework is based on information theory and utilizes a Markovian language model derived from linguistic analysis of language corpora for English, French, and German. For effective word repetition a selection controller adapts to the user's state and optimizes the learning process by minimizing error entropy. The performance of the method was evaluated in a large scale experimental study involving 80 dyslexic and non-dyslexic children. The results show significant improvements in writing skills in both groups after small amounts of daily training. Our approach combines findings from 3D computer graphics, visualization, linguistics, perception, psychology, and information theory.

A Greedy Renormalization Method for Arithmetic Coding

A typical arithmetic coder consists of three steps: range calculation, renormalization, and probability model updating. In this paper, we propose and analyze from an information theoretic point of view a greedy renormalization method, which has two components: greedy thresholding and greedy outputting. The method significantly reduces the computational complexity of the renormalization step of arithmetic coding by (1) using the greedy thresholding to minimize the number of renormalizations required to encode a sequence and (2) using the greedy outputting to minimize the number of operations within each renormalization. The method is particularly suitable for binary arithmetic coding (BAC). Two BAC algorithms based on this method are presented. The first algorithm replaces the renormalization method in the TOIS BAC with the greedy renormalization method, and keeps other parts of the TOIS BAC unchanged. For binary independent and identically distributed (i.i.d.) sources with the probability of the less probable symbol ranging from --, over gain in speed (on average), and less than loss in compression rate (in the worst case) are observed in the experiments. The second algorithm combines the greedy renormalization method with the QM-Coder. On an average, gain in speed and gain in compression rate are observed in the experiments.

Performance evaluation of a new BPSK-OFDM timing estimation algorithm

Previously, the timing estimation in OFDM system employs cyclic prefix has been presented with the assumption of independent identified distributed OFDM symbol. The information used to find the symbol synchronization depends on the length of cyclic prefix. Actually, the data in the BPSK-OFDM modulated symbol, transferred by inverse fast Fourier transform, is with a complex symmetry character. With this character, more information from the whole OFDM symbol could be provided for the symbol timing. In this paper, the proposed algorithm uses the autocorrelation of the received signal to obtain the symbol timing. The simulation of Beek's algorithm and the proposed scheme are presented. The performances of these algorithms are evaluated based on the missing symbol probability and the estimator mean square error. The simulation results show that the proposed algorithm is suitable to achieve a better symbol synchronization.

Gaussian Approximation Based Mixture Reduction for Joint Channel Estimation and Detection in MIMO Systems

A novel Gaussian approximation based mixture reduction algorithm is proposed for semi-blind joint channel tracking and symbol detection for spatial multiplexing multiple-input multiple-output (MIMO) systems with frequency-flat time-selective channels. The proposed algorithm is based on a modified sequential Gaussian approximation detector (SGA) which takes into account channel uncertainty, and the first order generalized pseudo-Bayesian (GPB1) channel estimator. Simulation results show that the proposed algorithm performs better than the conventional and computationally expensive decision-directed method with Kalman filter based channel estimation and a posteriori probability (APP) symbol detection.

New Bounds on the Expected Length of Optimal One-to-One Codes

In this correspondence, we consider one-to-one encodings for a discrete memoryless source, which are "one-shot" encodings associating a distinct codeword with each source symbol. Such encodings could be employed when only a single source symbol rather than a sequence of source symbols needs to be transmitted. For example, such a situation can arise when the last message must be acknowledged before the next message can be transmitted. We consider two slightly different types of one-to-one encodings (depending on whether the empty codeword is used or not) and obtain lower and upper bounds on the expected length of optimal one-to-one codes. We first give an extension of a known tight lower bound on the expected length of optimal one-to-one codes for the case that the the size of the source alphabet is finite and partial information about the source symbol probabilities is available. As expected, our lower bound is no less than the previously known lower bound obtained without side information about the source symbol probabilities. We then consider the case that the source entropy is available and derive arbitrarily tight lower bounds on the expected length of optimal one-to-one codes. We also derive arbitrarily tight lower bounds for the case that the source entropy and the probability of the most likely source symbol are available. Finally, given that the probability of the most likely source symbol is available, we obtain an upper bound on the expected length of optimal one-to-one codes. Our upper bound is tighter than the best upper bound known in the literature

Parallel Decoding of Context-Based Adaptive Binary Arithmetic Codes Based on Most Probable Symbol Prediction

Context-based adaptive binary arithmetic coding (CABAC) is the major entropy-coding algorithm employed in H.264/AVC. Although the performance gain of H.264/AVC is mainly due to CABAC, it is difficult to achieve a fast decoder because the decoding algorithm is basically sequential and computationally intensive. In this letter, a prediction scheme is proposed that enhances overall decoding performance by decoding two binary symbols at a time. A CABAC decoder based on the proposed prediction scheme improves the decoding performance by 24% compared to conventional decoders.

Channel and Data Estimation for Ad Hoc Networks and Cognitive Radio

Estimation of channel and data characteristics by the receiver is important in adaptive wireless transmission protocols and in cognitive radio. This paper formulates the estimation problem with the help of an illustrative example from the IEEE 802.11a OFDM standard. The problem reduces to the estimation of the common component variance and mixing probabilities in a finite Gaussian mixture, with known values for component means. Using the known component means, μ1, ... , μ M , a set of non-linear transformations, $${\sinh \mu_i x}$$ and $${\cosh \mu_i x}$$ of the data (mixture random variable X) are used to develop convergent and computationally efficient estimators for both the noise variance and the vector of symbol probabilities. The estimation equations can be implemented recursively or with a batch processing algorithm. Asymptotic variances of the estimates and the Cramer–Rao minimum variance bounds are derived. The estimates converge to true unknowns even when the sequences of noise and data symbols are dependent sequences. The OFDM example is simulated with parameters corresponding to the highest acceptable error rate. For a time-varying channel model chosen from the literature, it is shown that our estimator receives considerably more than adequate amount of data during an average time interval of unchanging channel characteristics. Analytical results, numerical results and related issues are discussed.

Distributed Turbo Coding With Soft Information Relaying in Multihop Relay Networks

It has been shown that distributed turbo coding (DTC) can approach the capacity of a wireless relay network. In the existing DTC schemes, it is usually assumed that error-free decoding is performed at a relay. We refer to this type of DTC schemes as perfect DTC. In this paper, we propose a novel DTC scheme. For the proposed scheme, instead of making a decision on the transmitted information symbols at the relay as in perfect DTC, we calculate and forward the corresponding soft information. We derive parity symbol soft estimates for the interleaved source information when only the a posteriori probabilities of the information symbols are known. The results show that the proposed scheme can effectively mitigate error propagation due to erroneous decoding at the relay. Simulation results also confirm that the proposed scheme approaches the outage probability bound of a distributed two-hop relay network at high signal-to-noise ratios

Stochastic Automata-Based Estimators for Adaptively Compressing Files With Nonstationary Distributions

This correspondence shows that learning automata techniques, which have been useful in developing weak estimators, can be applied to data compression applications in which the data distributions are nonstationary. The adaptive coding scheme utilizes stochastic learning-based weak estimation techniques to adaptively update the probabilities of the source symbols, and this is done without resorting to either maximum likelihood, Bayesian, or sliding-window methods. The authors have incorporated the estimator in the adaptive Fano coding scheme and in an adaptive entropy-based scheme that "resembles" the well-known arithmetic coding. The empirical results obtained for both of these adaptive methods are obtained on real-life files that possess a fair degree of nonstationarity. From these results, it can be seen that the proposed schemes compress nearly 10% more than their respective adaptive methods that use maximum-likelihood estimator-based estimates.

Parallelization of Context-Based Adaptive Binary Arithmetic Coders

This paper presents novel approaches to parallelize context-based adaptive binary arithmetic coders (CABACs). Two new parallelized CABACs (or PCABACs) are presented and the methods described. These coders are designed by modifying commonly used binary multiplication-free arithmetic coders. One utilizes linear approximation and simplifies the hardware by assuming that the less probable symbol probability is almost the same while performing the en/decoding (referred to as QL-coder). Another codec applies table lookup technique and achieves parallelism with a parallelized probability model (referred to as QT-coder). QL-coder is improved from the IBM Q-coder, and the QT-coder is improved from the CABAC used in H.264 video compression standard. Throughput, in both coders, is significantly increased after parallelization. A fast interval search method is also proposed

A Co-channel Interference Cancellation Method Using Low Dimensional Sphere Decoding for MIMO Communication Systems

This paper proposes a co-channel interference cancellation method for multiple-input multiple-output (MIMO) wireless communication systems. Maximum-likelihood multi-user detection (ML-MUD), which is one of the co-channel interference cancellation methods at a receiver side, has excellent bit error rate (BER) performance. However, computational complexity of the ML-MUD is prohibitive, because the ML-MUD must search for the most probable symbol vector from all candidates of the transmitted signals. We apply sphere decoding (SD) to the ML-MUD in order to reduce the computational complexity of the ML-MUD, and moreover we propose a modified version of the SD suitable for the ML-MUD. The proposed method extracts desired signal components from a received signal vector and a channel matrix decomposed the upper triangular form, and then performs the SD to the low dimensional model in order to detect the transmitted signals of the desired user. Computer simulation confirms that the proposed method can suppress the undesired signals and detect the desired signals, offering significant reduction of the computational complexity of the conventional method.

Advanced detection and coding techniques for nonlinear intersymbol interference cancellation in 40 Gb/s systems

To mitigate the fibre nonlinearities in high-speed transmission (at 40 Gb/s or higher), the use of maximum a posteriori probability (MAP) symbol decoding supplemented with iterative decoding is proposed. The MAP detector operates on the channel trellis to correct the corrupted data and to provide soft outputs processed further in an iterative decoder. Impressive performance improvement is demonstrated.