Finite Precision Representation Research Articles

Strengthening Quality of Chaotic Bit Sequences

We discuss chaos and its quality as measured through the 0-1 test for chaos. When the 0-1 test indicates deteriorating quality of chaos, because of the finite precision representations of real numbers in digital implementations, then the process may eventually lead to a periodic sequence. A simple method for improving the quality of a chaotic signal is to mix the signal with another signal by using the XOR operation. In this paper, such mixing of weak chaotic signals is considered, yielding new signals with improved quality (with K values from the 0-1 test close to 1). In some sense, such a mixing of signals could be considered as a two-layer prevention strategy to maintain chaos. That fact may be important in those applications when the hardware resources are limited. The 0-1 test is used to show the improved chaotic behavior in the case when a continuous signal (for example, from the Chua, Rössler or Lorenz system) intermingles with a discrete signal (for example, from the logistic, Tinkerbell or Henon map). The analysis is presented for chaotic bit sequences. Our approach can further lead to hardware applications, and possibly, to improvements in the design of chaotic bit generators. Several illustrative examples are included.

Wishart planted ensemble: A tunably rugged pairwise Ising model with a first-order phase transition.

We propose the Wishart planted ensemble, a class of zero-field Ising models with tunable algorithmic hardness and specifiable (or planted) ground state. The problem class arises from a simple procedure for generating a family of random integer programming problems with specific statistical symmetry properties but turns out to have intimate connections to a sign-inverted variant of the Hopfield model. The Hamiltonian contains only 2-spin interactions, with the coupler matrix following a type of Wishart distribution. The class exhibits a classical first-order phase transition in temperature. For some parameter settings the model has a locally stable paramagnetic state, a feature which correlates strongly with difficulty in finding the ground state and suggests an extremely rugged energy landscape. We analytically probe the ensemble thermodynamic properties by deriving the Thouless-Anderson-Palmer equations and free energy and corroborate the results with a replica and annealed approximation analysis; extensive Monte Carlo simulations confirm our predictions of the first-order transition temperature. The class exhibits a wide variation in algorithmic hardness as a generation parameter is varied, with a pronounced easy-hard-easy profile and peak in solution time towering many orders of magnitude over that of the easy regimes. By deriving the ensemble-averaged energy distribution and taking into account finite-precision representation, we propose an analytical expression for the location of the hardness peak and show that at fixed precision, the number of constraints in the integer program must increase with system size to yield truly hard problems. The Wishart planted ensemble is interesting for its peculiar physical properties and provides a useful and analytically transparent set of problems for benchmarking optimization algorithms.

Quantized compressive sensing with RIP matrices: the benefit of dithering

Abstract Quantized compressive sensing deals with the problem of coding compressive measurements of low-complexity signals with quantized, finite precision representations, i.e., a mandatory process involved in any practical sensing model. While the resolution of this quantization impacts the quality of signal reconstruction, there exist incompatible combinations of quantization functions and sensing matrices that proscribe arbitrarily low reconstruction error when the number of measurements increases. This work shows that a large class of random matrix constructions known to respect the restricted isometry property (RIP) is ‘compatible’ with a simple scalar and uniform quantization if a uniform random vector, or a random dither, is added to the compressive signal measurements before quantization. In the context of estimating low-complexity signals (e.g., sparse or compressible signals, low-rank matrices) from their quantized observations, this compatibility is demonstrated by the existence of (at least) one signal reconstruction method, the projected back projection, whose reconstruction error decays when the number of measurements increases. Interestingly, given one RIP matrix and a single realization of the dither, a small reconstruction error can be proved to hold uniformly for all signals in the considered low-complexity set. We confirm these observations numerically in several scenarios involving sparse signals, low-rank matrices and compressible signals, with various RIP matrix constructions such as sub-Gaussian random matrices and random partial discrete cosine transform matrices.

Preserving differential privacy under finite-precision semantics

The approximation introduced by finite-precision representation of continuous data can induce arbitrarily large information leaks even when the computation using exact semantics is secure. Such leakage can thus undermine design efforts aimed at protecting sensitive information. We focus here on differential privacy, an approach to privacy that emerged from the area of statistical databases and is now widely applied also in other domains. In this approach, privacy is protected by adding noise to the values correlated to the private data. The typical mechanisms used to achieve differential privacy have been proved correct in the ideal case in which computations are made using infinite-precision semantics. In this paper, we analyze the situation at the implementation level, where the semantics is necessarily limited by finite precision, i.e., the representation of real numbers and the operations on them are rounded according to some level of precision. We show that in general there are violations of the differential privacy property, and we study the conditions under which we can still guarantee a limited (but, arguably, acceptable) variant of the property, under only a minor degradation of the privacy level.

Harmonics Measurement With a Modulated Sliding Discrete Fourier Transform Algorithm

Accurate harmonics estimation has become a key issue in power quality assessment. This paper deals with a discrete Fourier transform (DFT)-based measurement technique, which can be easily employed to accurately determine the harmonic components of a distorted signal, i.e., voltage or current. The proposed method is based on a modulated sliding DFT algorithm, which is unconditionally stable and does not accumulate errors due to finite precision representation, and a variable sampling period technique (VSPT) to achieve a frequency adaptive mechanism. It is worth noting that the VSPT changes the sampling period for a variable grid frequency condition, leading to a constant sampling frequency under steady-state conditions. The proposed method provides: 1) high degree of accuracy; 2) structural/performance robustness; and 3) frequency adaptability. Given the modular nature of the method, it is implemented on a field programmable gate array. Simulations and experimental tests are shown to verify the performance of the proposed method.

Preserving differential privacy under finite-precision semantics.

The approximation introduced by finite-precision representation of continuous data can induce arbitrarily large information leaks even when the computation using exact semantics is secure. Such leakage can thus undermine design efforts aimed at protecting sensitive information. We focus here on differential privacy, an approach to privacy that emerged from the area of statistical databases and is now widely applied also in other domains. In this approach, privacy is protected by the addition of noise to a true (private) value. To date, this approach to privacy has been proved correct only in the ideal case in which computations are made using an idealized, infinite-precision semantics. In this paper, we analyze the situation at the implementation level, where the semantics is necessarily finite-precision, i.e. the representation of real numbers and the operations on them, are rounded according to some level of precision. We show that in general there are violations of the differential privacy property, and we study the conditions under which we can still guarantee a limited (but, arguably, totally acceptable) variant of the property, under only a minor degradation of the privacy level. Finally, we illustrate our results on two cases of noise-generating distributions: the standard Laplacian mechanism commonly used in differential privacy, and a bivariate version of the Laplacian recently introduced in the setting of privacy-aware geolocation.

On the Overflow Problem in Finite Precision Turbo Decoding Message Passing

Much attention has been paid recently to the so-called layered decoding of LDPC codes, also known as turbo-decoding message passing (TDMP). The TDMP algorithm for decoding LDPC codes is known to possess some desirable features, such as fast convergence speed, reduced memory requirements and reduced implementation complexity in comparison with standard two-phase message passing algorithm. In this paper we analyze an important issue connected with hardware implementation of TDMP algorithm, namely the finite precision representation of messages influence on the decoding performance. Constrained dynamic range of the finite precision representation of messages entails overflow errors. We present an analysis revealing that in the subsequent decoding iterations, the subtraction of non-overflowed intrinsic message from overflowed extrinsic message is a source of errors that have substantial impact on the decoding results. The analysis is confirmed by simulation results showing significant performance loss. However this performance loss can be almost completely eliminated with a basic modifications in the messages computation algorithm. Effectiveness of the presented modifications is confirmed by simulation results obtained with hardware TDMP decoder implementation that has been developed.

Numerical Verification of Industrial Numerical Codes

Several approximations occur during a numerical simulation: physical effects mapy be discarded, continuous functions replaced by discretized ones and real numbers replaced by finite-precision representations. The use of the floating point arithmetic generates round-off errors at each arithmetical expression and some mathematical properties are lost. The aim of the numerical verification activity at EDF R&D is to study the effect of the round-off error propagation on the results of a numerical simulation. It is indeed crucial to perform a numerical verification of industrial codes such as developped at EDF R&D even more for code running in HPC environments. This paper presents some recent studies around the numerical verification at EDF R&D.

Power optimised channel coding in wireless sensor networks using low-density parity-check codes

Energy consumption of low-density parity-check (LDPC) codes in different implementations is evaluated. Decoder's complexity is reduced by finite precision representation of messages, that is, quantised LDPC decoder, and replacement of function blocks with look-up tables. It is shown that the decoder's energy consumption increases exponentially with the number of quantisation bits. For the sake of low-power consumption, 3-bit magnitude and 1-sign bit representation for messages are used in the decoder. It is concluded that high-rate Gallager codes are as energy efficient as the Reed-Solomon codes, which till now have been the first choice for wireless sensor networks (WSNs). Finally, it is shown that using LDPC codes in WSNs can be justified even more by applying the idea of trading the transmitter power with the decoder energy consumption. By exploiting the trade-off inherent in iterative decoding, the network lifetime is increased up to four times with the 3-6 regular LDPC code. Hence, it is inferred that the LDPC codes are more efficient than the block and the convolutional codes.

Mumford and Shah functional: VLSI analysis and implementation

This paper describes the analysis of the Mumford and Shah functional from the implementation point of view. Our goal is to show results in terms of complexity for real-time applications, such as motion estimation based on segmentation techniques, of the Mumford and Shah functional. Moreover, the sensitivity to finite precision representation is addressed, a fast VLSI architecture is described, and results obtained for its complete implementation on a 0.13 microm standard cells technology are presented.

A fast image encryption system based on chaotic maps with finite precision representation

Abstract In this paper, a fast chaos-based image encryption system with stream cipher structure is proposed. In order to achieve a fast throughput and facilitate hardware realization, 32-bit precision representation with fixed point arithmetic is assumed. The major core of the encryption system is a pseudo-random keystream generator based on a cascade of chaotic maps, serving the purpose of sequence generation and random mixing. Unlike the other existing chaos-based pseudo-random number generators, the proposed keystream generator not only achieves a very fast throughput, but also passes the statistical tests of up-to-date test suite even under quantization. The overall design of the image encryption system is to be explained while detail cryptanalysis is given and compared with some existing schemes.

Finite precision implementation of LDPC decoders

Low-density parity-check (LDPC) codes are among the most powerful known error correcting codes and they have been shown to achieve performance very close to the Shannon bound. In practical hardware implementations of LDPC decoders, bit error rate performances are affected strongly by fixed-point representation of processed data and, although many papers have been published in recent years on the subject of LDPC codes, only a few of them report results on the effects of finite precision. In the paper two novel arithmetic approximations for the basic processing operations of the decoding algorithm are presented. The two proposed solutions are analysed in terms of bit error rate performance for different finite precision representations of both external and internal data; the estimated performance loss is comparable to that reported by previous works and limited to approximately 0.1 dB for cases of interests. However the proposed solutions support a reduced complexity formulation of the decoding algorithm, so enabling the implementation of flexible decoders that can be adapted to different LDPC codes, block sizes and code rates.

Selecting accurate, robust, and minimal feedforward neural networks

Accuracy, robustness, and minimality are fundamental issues in system-level design. Such properties are generally associated with constraints limiting the feasible model space. The paper focuses on the optimal selection of feedforward neural networks under the accuracy, robustness, and minimality constraints. Model selection, with respect to accuracy, can be carried out within the theoretical framework delineated by the final prediction error (FPE), generalization error estimate (GEN), general prediction error (GPE) and network information criterion (NIC) or cross-validation-based techniques. Robustness is an appealing feature since a robust application provides a graceful degradation in performance once affected by perturbations in its structural parameters (e.g., associated with faults or finite precision representations). Minimality is related to model selection and attempts to reduce the computational load of the solution (with also silicon area and power consumption reduction in a digital implementation). A novel sensitivity analysis derived by the FPE selection criterion is suggested in the paper to quantify the relationship between performance loss and robustness; based on the definition of weak and acute perturbations, we introduce two criteria for estimating the robustness degree of a neural network. Finally, by ranking the features of the obtained models we identify the best constrained neural network.

A probably approximately correct framework to estimate performance degradation in embedded systems

Future design environments for embedded systems will require the development of sophisticated computer-aided design tools for compiling the high-level specifications of an application down to a final low-level language describing the embedded solution. This requires abstraction of technology-dependent aspects and requirements into behavioral entities. The paper takes a first step in this direction by introducing a high-level methodology for estimating the performance degradation of an application affected by perturbations; a special emphasis is given to accuracy performance. To grant generality it is uniquely assumed that the performance degradation function and the mathematical formulation describing the application are Lebesgue measurable. Perturbations affecting the application abstract details related to physical sources of uncertainties such as finite precision representation, faults, fluctuations of physical parameters, battery power variations, and aging effects whose impact on the computation can be treated within a high-level homogenous framework. A novel stochastic theory based on randomization is suggested to quantify the approximated nature of the perturbed environment. The outcomes are two algorithms which estimate in polynomial time the performance degradation of the application once affected by perturbations. Such information can then be exploited by HW/SW codesign methodologies to guide the subsequent partitioning between HW and SW, analog versus digital, fixed versus floating point, or used to validate architectural choices before any low-level design step takes place. The proposed method is finally applied to real designs involving neural and wavelet-based applications.

Randomized algorithms: a system-level, poly-time analysis of robust computation

Provides a methodology for analyzing the performance degradation of a computation once it has been affected by perturbations. The suggested methodology, by relaxing all assumptions made in the related literature, provides design guidelines for the subsequent implementation of complex computations in physical devices. Implementation issues, such as finite precision representation, fluctuations of the production parameters and aging effects, can be studied directly at the system level, independent of any technological aspect and quantization technique. Only the behavioral description of the computational flow, which is assumed to be Lebesgue-measurable, and the architecture to be investigated are needed. The suggested analysis is based on the theory of randomized algorithms, which transforms the computationally intractable problem of robustness investigation into a polynomial-time algorithm by resorting to probability.

Optimization and implementation of the integer wavelet transform for image coding

This paper deals with the design and implementation of an image transform coding algorithm based on the integer wavelet transform (IWT). First of all, criteria are proposed for the selection of optimal factorizations of the wavelet filter polyphase matrix to be employed within the lifting scheme. The obtained results lead to the IWT implementations with very satisfactory lossless and lossy compression performance. Then, the effects of finite precision representation of the lifting coefficients on the compression performance are analyzed, showing that, in most cases, a very small number of bits can be employed for the mantissa keeping the performance degradation very limited. Stemming from these results, a VLSI architecture is proposed for the IWT implementation, capable of achieving very high frame rates with moderate gate complexity.

Finite Precision Representation of the Conley Decomposition

We present a theoretical basis for a novel way of studying and representing the long-time behavior of finite-dimensional maps. It is based on a finite representation of e-pseudo orbits of a map by the sample paths of a suitable Markov chain based on a finite partition of phase space. The use of stationary states of the chain and the corresponding partition elements in approximating the attractors of maps and differential equations was demonstrated in Refs. 7 and 3 and proved for a class of stable attracting sets in Ref. 8. Here we explore the relationship between the communication classes of the Markov chain and basic sets of the Conley decomposition of a dynamical system. We give sufficient conditions for the existence of a chain transitive set and show that basic sets are isolated from each other by neighborhoods associated with closed communication classes of the chain. A partition element approximation of an isolating block is introduced that is easy to express in terms of sample paths. Finally, we show that when the map supports an SBR measure there is a unique closed communication class and the Markov chain restricted to those states is irreducible.

The quantization effects of the CORDIC algorithm

A detailed analysis of the quantization error encountered in the CORDIC (coordinate rotation digital computer) algorithm is presented. Two types of quantization error are examined: an approximation error due to the quantized representation of rotation angles, and a rounding error due to the finite precision representation in both fixed-point and floating-point arithmetic. Tight error bounds for these two types of error are derived. The rounding error due to a scaling (normalization) operation in the CORDIC algorithm is also discussed. An expression for overall quantization error is derived, and several simulation examples are presented. >

On finite-precision representations of geometric objects

The paper presents an experimental study on approximation of continuously distributed geometric objects by equations with integer coefficients. The problem considered here is, given an equation of a geometric object with real coefficients, to find a set of integer coefficients that gives a good approximation to the original equation. This problem is reduced to a problem called a simultaneous Diophantine approximation problem, for which no efficient method to give the best solution has yet been found. Here considered are three heuristic methods: a naive method, a method based on the continued fraction theory, and a method using the basis reduction technique. These methods together with a brute-force exhaustive search method are studied using randomly generated data, and also proposed is a hybrid method, which seems allowable from both the viewpoint of time complexity and the viewpoint of quality of approximation.