Uniform Input Distribution Research Articles

Information transfer rate in BCIs: Towards tightly integrated symbiosis

Objective:The information transmission rate (ITR), or effective bit rate, is a popular and widely used information measurement metric, particularly popularized for SSVEP-based Brain–Computer (BCI) interfaces. By combining speed and accuracy into a single-valued parameter, this metric aids in the evaluation and comparison of various target identification algorithms across different BCI communities. In order to calculate ITR, it is customary to assume a uniform input distribution and an oversimplified channel model that is memoryless, stationary, and symmetrical in nature with discrete alphabet sizes. To accurately depict performance and inspire an end-to-end design for futuristic BCI designs, a more thorough examination and definition of ITR is therefore required. Approach:We model the symbiotic communication medium, hosted by the retinogeniculate visual pathway, as a discrete memoryless channel and use the modified capacity expressions to redefine the ITR. We leverage a result for directed graphs to characterize the relationship between the asymmetry of the transition statistics and the ITR gain due to the new definition, leading to potential bounds on data rate performance. Main Results:On two well-known SSVEP datasets, we compared two cutting-edge target identification methods. Results indicate that the induced DM channel asymmetry has a greater impact on the actual perceived ITR than the change in input distribution. Moreover, it is demonstrated that the ITR gain under the new definition is inversely correlated with the asymmetry in the channel transition statistics. Individual input customizations are further shown to yield perceived ITR performance improvements. Finally, an algorithm is proposed to find the capacity of binary classification and further discussions are given to extend such results to multi-class case through ensemble techniques. Significance:We anticipate that the results of our study will contribute to the characterization of the highly dynamic BCI channel capacities, performance thresholds, and improved BCI stimulus designs for a tighter symbiosis between the human brain and computer systems while ensuring the efficient utilization of the underlying communication resources.

Design of Generalized Enhanced Static Segment Multiplier with Minimum Mean Square Error for Uniform and Nonuniform Input Distributions

In this paper, we analyze the performances of an Enhanced Static Segment Multiplier (ESSM) when the inputs have both uniform and non-uniform distribution. The enhanced segmentation divides the multiplicands into a lower, a middle, and an upper segment. While the middle segment is placed at the center of the inputs in other implementations, we seek the optimal position able to minimize the approximation error. To this aim, two design parameters are exploited: m, defining the size and the accuracy of the multiplier, and q, defining the position of the middle segment for further accuracy tuning. A hardware implementation is proposed for our generalized ESSM (gESSM), and an analytical model is described, able to find m and q which minimize the mean square approximation error. With uniform inputs, the error slightly improves by increasing q, whereas a large error decrease is observed by properly choosing q when the inputs are half-normal (with a NoEB up to 18.5 bits for a 16-bit multiplier). Implementation results in 28 nm CMOS technology are also satisfactory, with area and power reductions up to 71% and 83%. We report image and audio processing applications, showing that gESSM is a suitable candidate in applications with non-uniform inputs.

On capacity and performance of noncoherent OFDM-IM systems with binary codes

This paper studies the capacity and error performance of a noncoherent index-modulated orthogonal frequency division multiplexing (OFDM-IM) system with binary coded subcarrier activation patterns. To that end, soft-decision (i.e., unquantized) and hard-decision (i.e., quantized) channels are considered, where the latter is equivalent to a binary asymmetric channel (BAC). By using maximum likelihood and maximum a posteriori criteria, crossover probabilities of the quantized channel are derived in order to characterize its capacity and probability of error in detecting IM signals. Capacity expressions are then given for both channels. Using the Blahut-Arimoto algorithm, an iterative algorithm is developed to find the optimal input distribution for the unquantized channel, while for the quantized channel, it is found by solving a nonlinear equation for its zero. It is shown that using a uniform input distribution instead of the optimal (nonuniform) one results in a negligible capacity loss, therefore binary codes with a uniform distribution of 1s and 0s are nearly optimal. Compared to the unquantized channel, it is shown that an approximate loss less than 2dB is incurred when quantization is used and this loss diminishes rapidly as signal-to-noise ratio (SNR) increases. In addition, it is demonstrated that the gap (in dB) between uncoded noncoherent OFDM-IM system and the capacity limit is generally wide, and the gap narrows when subcarrier clustering is considered. The findings in this paper can have theoretical and practical significance for understanding capacity limits and can provide some guidance for coding design in noncoherent OFDM-IM systems.

Convergence analysis for gradient flows in the training of artificial neural networks with ReLU activation

Gradient descent (GD) type optimization schemes are the standard methods to train artificial neural networks (ANNs) with rectified linear unit (ReLU) activation. Such schemes can be considered as discretizations of the corresponding gradient flows (GFs). In this work we analyze GF processes in the training of ANNs with ReLU activation and three layers. In particular, in this article we prove in the case where the distribution of the input data is absolutely continuous with respect to the Lebesgue measure that the risk of every bounded GF trajectory converges to the risk of a critical point. In addition, we show in the case of a 1-dimensional affine target function and a uniform input distribution that the risk of every bounded GF trajectory converges to zero if the initial risk is sufficiently small. Finally, we show that the boundedness assumption can be removed if the hidden layer consists of only one neuron.

A Cognitive Bias for Zipfian Distributions? Uniform Distributions Become More Skewed via Cultural Transmission

Abstract There is growing evidence that cognitive biases play a role in shaping language structure. Here, we ask whether such biases could contribute to the propensity of Zipfian word-frequency distributions in language, one of the striking commonalities between languages. Recent theoretical accounts and experimental findings suggest that such distributions provide a facilitative environment for word learning and segmentation. However, it remains unclear whether the advantage found in the laboratory reflects prior linguistic experience with such distributions or a cognitive preference for them. To explore this, we used an iterated learning paradigm—which can be used to reveal weak individual biases that are amplified overtime—to see if learners change a uniform input distribution to make it more skewed via cultural transmission. In the first study, we show that speakers are biased to produce skewed word distributions in telling a novel story. In the second study, we ask if this bias leads to a shift from uniform distributions towards more skewed ones using an iterated learning design. We exposed the first learner to a story where six nonce words appeared equally often, and asked them to re-tell it. Their output served as input for the next learner, and so on for a chain of ten learners (or ‘generations’). Over time, word distributions became more skewed (as measured by lower levels of word entropy). The third study asked if the shift will be less pronounced when lexical access was made easier (by reminding participants of the novel word forms), but this did not have a significant effect on entropy reduction. These findings are consistent with a cognitive bias for skewed distributions that gets amplified over time and support the role of entropy minimization in the emergence of Zipfian distributions.

Discussion: “Bayesian Optimal Design of Experiments for Inferring the Statistical Expectation of Expensive Black-Box Functions” (Pandita, P., Bilionis, I., and Panchal, J., 2019, ASME J. Mech. Des., 141(10), p. 101404)

Abstract The authors of the discussed paper simplified the information-based acquisition on estimating statistical expectation and developed analytical computation for each involved quantity under uniform input distribution. In this discussion, we show that (1) the last three terms of the acquisition always add up to zero, leaving a concise form with a much more intuitive interpretation of the acquisition; (2) the analytical computation of the acquisition can be generalized to arbitrary input distribution, greatly broadening the application of the developed framework.

Channel Capacity of an Asymmetric Constellation in Rayleigh Fading With Noncoherent Energy Detection

The channel capacity of noncoherent reception of multi-level one-sided amplitude-shift keying (ASK), which is an asymmetric constellation, in Rayleigh fading with receive diversity and energy detection is considered. The asymmetries result in capacity achieving input probability distributions, that is, a priori probability distributions, that deviate from uniformity. An analytical expression for the mutual information in terms of a single integral is derived, and from it the set of equations, which can be solved to obtain the optimum or capacity achieving input probabilities, is obtained. High and low signal-to-noise ratio (SNR) approximations of the optimum input probabilities and the capacity are derived next. Furthermore, a logarithmic upper bound on the mutual information is obtained. Numerical results confirm that the uniform distribution of input probabilities is not capacity achieving. For example, with average SNR per symbol per branch of 6 dB, the relative deviation of the mutual information (with uniform input distribution) from the capacity is nearly 20% for 4-level ASK with one transmit diversity branch and two receive diversity branches. Furthermore, the derived high and low SNR approximations to the capacity are shown to be reasonably accurate.

High-dimensional distribution generation through deep neural networks

We show that every d-dimensional probability distribution of bounded support can be generated through deep ReLU networks out of a 1-dimensional uniform input distribution. What is more, this is possible without incurring a cost—in terms of approximation error measured in Wasserstein-distance—relative to generating the d-dimensional target distribution from d independent random variables. This is enabled by a vast generalization of the space-filling approach discovered in Bailey and Telgarsky (in: Bengio (eds) Advances in neural information processing systems vol 31, pp 6489–6499. Curran Associates, Inc., Red Hook, 2018). The construction we propose elicits the importance of network depth in driving the Wasserstein distance between the target distribution and its neural network approximation to zero. Finally, we find that, for histogram target distributions, the number of bits needed to encode the corresponding generative network equals the fundamental limit for encoding probability distributions as dictated by quantization theory.

Point Transformer for Shape Classification and Retrieval of Urban Roof Point Clouds

The success of deep learning methods led to significant breakthroughs in 3-D point cloud processing tasks with applications in remote sensing. Existing methods utilize convolutions that have some limitations, as they assume a uniform input distribution and cannot learn long-range dependences. Recent works have shown that adding attention in conjunction with these methods improves performance. This raises a question: can attention layers completely replace convolutions? This letter proposes a fully attentional model—Point Transformer (PT) for deriving a rich point cloud representation. The model’s shape classification and retrieval performance are evaluated on a large-scale urban data set—RoofN3D and a standard benchmark data set ModelNet40. Extensive experiments are conducted to test the model’s robustness to unseen point corruptions for analyzing its effectiveness on real data sets. The proposed method outperforms other state-of-the-art models in the RoofN3D data set, gives competitive results in the ModelNet40 benchmark, and shows high robustness to various unseen point corruptions. Furthermore, the model is highly memory and space-efficient when compared to other methods.

Achievable Rate and Capacity Analysis for Ambient Backscatter Communications

In this paper, we analyze the achievable rate for ambient backscatter communications under three different channels: the binary input and binary output (BIBO) channel, the binary input and signal output (BISO) channel, and the binary input and energy output (BIEO) channel. Instead of assuming Gaussian input distribution, the proposed study matches the practical ambient backscatter scenarios, where the input of the tag can only be binary. We derive the closed-form capacity expression as well as the capacity-achieving input distribution for the BIBO channel. To show the influence of the signal-to-noise ratio (SNR) on the capacity, a closed-form tight ceiling is also derived when SNR turns relatively large. For BISO and BIEO channel, we obtain the closed-form mutual information, while the semi-closed-form capacity value can be obtained via one dimensional searching. Simulations are provided to corroborate the theoretical studies. Interestingly, the simulations show that: (i) the detection threshold maximizing the capacity of BIBO channel is the same as the one from the maximum likelihood signal detection; (ii) the maximal of the mutual information of all channels is achieved almost by a uniform input distribution; and (iii) the mutual information of the BIEO channel is larger than that of the BIBO channel, but is smaller than that of the BISO channel.

IDrAx: A tool-chain for designing efficient approximate adders

As transistors shrink down, improving the energy-efficiency of nanometer ICs has gained a great deal of attention. Recently, approximate computing has been introduced to address energy-efficiency problems of error-tolerant systems. Adders are the most widely used arithmetic modules that consume significant amounts of system's total energy and area. The error-resiliency of these systems has made it possible to innovate energy-efficiency approximate adders to obtain low power and high-speed circuits. This paper presents IDrAx, an approach to improve the accuracy of approximate adders. The goal is to find the most significant errors in the error distribution for the given circuit and correct them through low-cost additional circuits. The proposed method is applied to several state-of-the-art approximate adders for evaluation by considering two different input distributions. Our simulation results indicate that the proposed design achieves by 75% and 78.01% accuracy regarding mean error distance metric for uniform and normal input distributions, respectively.

An Orthogonal Polynomial-Based Analytical Expression of Nonlinear Power Amplifier for FBMC Systems

A filter bank multicarrier modulation (FBMC) is a strong promising solution for the fifth generation (5G) and beyond in broadband wireless communications. Nonetheless, one of the main disadvantages of the multicarrier techniques-based communication systems is the high peak-to-average power ratio (PAPR) of transmitting signals resulting in a nonlinear distortion of the signals, which is caused by a high power amplifier (HPA). Many kinds of research related to the nonlinear distortions induced by HPA have been proposed. However, there are solely limited studies on the analytical derivation of the nonlinear HPA characteristic. In this paper, an orthogonal polynomial based-analytical expression of a nonlinear power amplifier for baseband uniform, exponential, and Rayleigh input distribution of FBMC systems is proposed. The probability of error performances (BER) comparison between the proposed orthogonal polynomial method and the conventional polynomial method is determined. In addition, a theoretical performance analysis of the proposed method is derived to evaluate the BER performances in the frequency-selective Rayleigh fading channel. In the simulation, the BER performance of a nonlinearly amplified FBMC systems is investigated at a different IBO. The simulation results indicate that the proposed orthogonal polynomial method significantly outperforms the conventional polynomial method in terms of a normalized mean squared error (NMSE) and BER performance. Moreover, the theoretical error probability is compared with the simulation results and it could be found that the theoretical BER performance is relatively close to the simulated result.

An Efficient Method for Calculating the Error Statistics of Block-Based Approximate Adders

Adders are key building blocks of many error-tolerant applications. Recently, a number of approximate adders were proposed. Many of them are block-based approximate adders. For approximate circuits, besides normal metrics such as area and delay, the other important design metrics are the various error statistics, such as error rate (ER), mean error distance (MED), and mean square error (MSE). Given the popularity of block-based approximate adders, in this work, we propose an efficient method to obtain their error statistics. We first show how to calculate the ER. Then, we demonstrate an approach to get the error distribution, which can be used to calculate other metrics, such as MED and MSE. Our method is applicable to an arbitrary block-based approximate adder. It is accurate for the uniformly distributed inputs. Experimental results also demonstrated that it produces error metrics close to the accurate ones for various types of non-uniform input distributions. Compared to the state-of-the-art algorithm for obtaining the error distributions of block-based approximate adders, for the uniform input distribution, our method improves the runtime by up to $4.8\times 10^4$ times with the same accuracy; for non-uniform input distributions, it achieves a speed-up of up to 400 times with very similar accuracy.

A Neural Architecture for Bayesian Compressive Sensing Over the Simplex via Laplace Techniques

This paper presents a theoretical and conceptual framework to design neural architectures for Bayesian compressive sensing of simplex-constrained sparse stochastic vectors. First, we recast the problem of MMSE estimation (w.r.t. a pre-defined uniform input distribution over the simplex) as the problem of computing the centroid of a polytope that is equal to the intersection of the simplex and an affine subspace determined by compressive measurements. Then, we use multidimensional Laplace techniques to obtain a closed-form solution to this computation problem, and we show how to map this solution to a neural architecture comprising threshold functions, rectified linear (ReLU) and rectified polynomial activation functions. In the proposed architecture, the number of layers is equal to the number of measurements that allows for faster solutions in the low-measurement regime when compared to the integration by domain decomposition or Monte Carlo approximation. We also show by simulation that the proposed solution is robust to small model mismatches; furthermore, the proposed architecture yields superior approximations with less parameters when compared to a standard ReLU architecture in a supervised learning setting.

Models and Information-Theoretic Bounds for Nanopore Sequencing

Nanopore sequencing is an emerging new technology for sequencing DNA, which can read long fragments of DNA (~50,000 bases) in contrast to most current short-read sequencing technologies which can only read hundreds of bases. While nanopore sequencers can acquire long reads, the high error rates (20%-30%) pose a technical challenge. In a nanopore sequencer, a DNA is migrated through a nanopore and current variations are measured. The DNA sequence is inferred from this observed current pattern using an algorithm called a base-caller. In this paper, we propose a mathematical model for the "channel" from the input DNA sequence to the observed current, and calculate bounds on the information extraction capacity of the nanopore sequencer. This model incorporates impairments like (non-linear) inter-symbol interference, deletions, as well as random response. These information bounds have two-fold application: (1) The decoding rate with a uniform input distribution can be used to calculate the average size of the plausible list of DNA sequences given an observed current trace. This bound can be used to benchmark existing base-calling algorithms, as well as serving a performance objective to design better nanopores. (2) When the nanopore sequencer is used as a reader in a DNA storage system, the storage capacity is quantified by our bounds.

Extremality Between Symmetric Capacity and Gallager’s Reliability Function $E_{0}$ for Ternary-Input Discrete Memoryless Channels

This paper examines the exact ranges between the symmetric capacity and Gallager’s reliability function $E_{0}$ for ternary-input discrete memoryless channels (T-DMCs) under a uniform input distribution. We first derive the two extremal ternary-input strongly symmetric channels taking the maximum and minimum values of the $E_{0}$ function among all ternary-input strongly symmetric channels with a fixed capacity. Extending the results of ternary-input strongly symmetric channels, we second derive the exact ranges between capacity and the $E_{0}$ function for ternary-input Gallager-symmetric channels. We third show that the exact ranges between the symmetric capacity and the $E_{0}$ function of T-DMCs coincide with the ranges of ternary-input Gallager-symmetric channels. In particular, we identify the extremal channels taking the maximum and minimum of $E_{0}$ among all T-DMCs with a fixed symmetric capacity. As applications of the results, we describe some bounds of error exponents for T-DMCs with a fixed symmetric capacity.

Perfect Secrecy in Physical-Layer Network Coding Systems From Structured Interference

Physical-layer network coding (PNC) has been proposed for next generation networks. In this paper, we investigate PNC schemes with embedded perfect secrecy by exploiting structured interference in relay networks with two users and a single relay. In a practical scenario where both users employ finite and uniform signal input distributions, we establish upper bounds (UBs) on the achievable perfect secrecy rates and make these explicit when pulse amplitude modulation modems are used. We then describe two simple, explicit encoders that can achieve perfect secrecy rates close to these UBs with respect to an untrustworthy relay in the single antenna and single relay setting. Last, we generalize our system to a multiple-input multiple-output relay channel, where the relay has more antennas than the users and study optimal precoding matrices, which maintain a required secrecy constraint. Our results establish that the design of PNC transmission schemes with enhanced throughput and guaranteed data confidentiality is feasible in next generation systems.

Asymmetric Evaluations of Erasure and Undetected Error Probabilities

The problem of channel coding with the erasure option is revisited for discrete memoryless channels. The interplay between the code rate, the undetected and total error probabilities is characterized. Using the information spectrum method, a sequence of codes of increasing blocklengths $n$ is designed to illustrate this tradeoff. Furthermore, for additive discrete memoryless channels with uniform input distribution, we establish that our analysis is tight with respect to the ensemble average. This is done by analyzing the ensemble performance in terms of a tradeoff between the code rate, the undetected and total errors. This tradeoff is parameterized by the threshold in a generalized likelihood ratio test. Two asymptotic regimes are studied. First, the code rate tends to the capacity of the channel at a rate slower than $n^{-1/2}$ corresponding to the moderate deviations regime. In this case, both error probabilities decay subexponentially and asymmetrically. The precise decay rates are characterized. Second, the code rate tends to capacity at a rate of $n^{-1/2}$ . In this case, the total error probability is asymptotically a positive constant, while the undetected error probability decays as $\exp (- bn^{ 1/2})$ for some $b>0$ . The proof techniques involve the applications of a modified (or shifted) version of the Gartner–Ellis theorem and the type class enumerator method to characterize the asymptotic behavior of a sequence of cumulant generating functions.

The VLT/NaCo large program to probe the occurrence of exoplanets and brown dwarfs at wide orbits

In anticipation of the VLT/SPHERE planet imager guaranteed time programs, we have conducted a preparatory survey of 86 stars between 2009 and 2013 in order to identify new faint comoving companions to ultimately carry out a comprehensive analysis of the occurence of giant planets and brown dwarf companions at wide (10-2000 AU) orbits around young, solar-type stars. We used NaCo at VLT to explore the occurrence rate of giant planets and brown dwarfs between typically 0.1 and 8''. Diffraction-limited observations in H-band combined with angular differential imaging enabled us to reach primary star-companion brightness ratios as small as 10-6 at 1.5''. 12 systems were resolved as new binaries, including the discovery of a new white dwarf companion to the star HD8049. Around 34 stars, at least one companion candidate was detected in the observed field of view. More than 400 faint sources were detected, 90% of them in 4 crowded fields. With the exception of HD8049B, we did not identify any new comoving companions. The survey also led to spatially resolved images of the thin debris disk around HD\,61005 that have been published earlier. Finally, considering the survey detection limits, we derive a preliminary upper limit on the frequency of giant planets for semi-major axes of [10,2000] AU: typically less than 15% between 100 and 500 AU, and less than 10% between 50 and 500 AU for exoplanets more massive than 5 MJup and 10 MJup respectively, considering a uniform input distribution and with a confidence level of 95%.

Extremal Channels of Gallager's $E_{0}$ Under the Basic Polarization Transformations

We study the extremality of the binary erasure channel and the binary symmetric channel for Gallager's reliability function E <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">0</sub> of binary input discrete memoryless channels evaluated under the uniform input distribution from the aspect of channel polarization. In particular, we show that amongst all binary discrete memoryless channels of a given E <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">0</sub> (ρ) value, for a fixed ρ ≥ 0, the binary erasure channel and the binary symmetric channel are extremal in the evolution of E <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">0</sub> under the one-step polarization transformations.