Approximate Minimum Mean Square Error Research Articles

Iterative algorithms and architectures in massive MIMO detection

Massive multiple-input multiple-output (MIMO) wireless systems play an important role in the 5G networks. The complexity of signal detection in massive MIMO is increasing rapidly due to the growth of the number of transmitting antennas. In this paper, we introduced different iterative algorithms to decrease the computational cost of the approximate minimum mean-square error (MMSE) algorithm, including the Neumann Series algorithm, Jacobi method, Gauss-Seidel method, Successive Over Relaxation method and the Conjugate Gradient method. In addition, the VLSI architecture implementations of algorithms mentioned above are also discussed in the article.

Communication-Efficient Federated Learning via Quantized Compressed Sensing

In this paper, we present a communication-efficient federated learning framework inspired by quantized compressed sensing. The presented framework consists of gradient compression for wireless devices and gradient reconstruction for a parameter server (PS). Our strategy for gradient compression is to sequentially perform block sparsification, dimensional reduction, and quantization. By leveraging both dimension reduction and quantization, our strategy can achieve a higher compression ratio than one-bit gradient compression. For accurate aggregation of local gradients from the compressed signals, we put forth an approximate minimum mean square error (MMSE) approach for gradient reconstruction using the expectation-maximization generalized-approximate-message-passing (EM-GAMP) algorithm. Assuming Bernoulli Gaussian-mixture prior, this algorithm iteratively updates the posterior mean and variance of local gradients from the compressed signals. We also present a low-complexity approach for the gradient reconstruction. In this approach, we use the Bussgang theorem to aggregate local gradients from the compressed signals, then compute an approximate MMSE estimate of the aggregated gradient using the EM-GAMP algorithm. We also provide a convergence rate analysis of the presented framework. Using the MNIST dataset, we demonstrate that the presented framework achieves almost identical performance with the case that performs no compression, while significantly reducing communication overhead for federated learning.

An approximate minimum mean-square error estimator for linear discrete time-varying systems: Handling Try-Once-Discard protocol

This paper is concerned with the remote state estimation problem for a class of linear discrete time-varying stochastic systems under communication constraints. In order to mitigate the undesired phenomena of data collisions, a Try-Once-Discard (TOD) protocol is utilized to regulate the signal transmissions over the sensor-to-estimator communication channel, where the TOD protocol is based on the scheduling rule described by a specific switching function. Under the introduced TOD protocol, the approximate minimum mean-square error (MMSE) state estimation problem is investigated and a recursive algorithm is developed for the MMSE estimator design whose computational complexity is comparable to that of the conventional Kalman filter. Furthermore, some conditions are established for the boundedness of the estimation error covariance. A numerical example is presented to illustrate the effectiveness of the proposed MMSE estimator.

Scheduler-based state estimation over multiple channels networks

We investigate the remote state estimation problem for networked systems over parallel noise-free communication channels. Due to limited network capabilities in practical network environments, communication schedulers are implemented at the transmit side of each subchannel to promote resource efficiency. Specifically, the processed signals are transmitted only when it is necessary to provide the real-time measurements to the remote estimator. The recursive approximate minimum mean-square error estimator is established to restore the state vector of a target plant by utilizing the scheduled transmission signals. All the information coming from the individual subchannels, even if no measurement is sent, will contribute to improve the estimation performance in an analytical form. Finally, a numerical example is given to illustrate the effectiveness of the main results.

Inference With Deep Generative Priors in High Dimensions

Deep generative priors offer powerful models for complex-structured data, such as images, audio, and text. Using these priors in inverse problems typically requires estimating the input and/or hidden signals in a multi-layer deep neural network from observation of its output. While these approaches have been successful in practice, rigorous performance analysis is complicated by the non-convex nature of the underlying optimization problems. This paper presents a novel algorithm, Multi-Layer Vector Approximate Message Passing (ML-VAMP), for inference in multi-layer stochastic neural networks. ML-VAMP can be configured to compute maximum a priori (MAP) or approximate minimum mean-squared error (MMSE) estimates for these networks. We show that the performance of ML-VAMP can be exactly predicted in a certain high-dimensional random limit. Furthermore, under certain conditions, ML-VAMP yields estimates that achieve the minimum (i.e., Bayes-optimal) MSE as predicted by the replica method. In this way, ML-VAMP provides a computationally efficient method for multi-layer inference with an exact performance characterization and testable conditions for optimality in the large-system limit.

Provable Convergence of Plug-and-Play Priors With MMSE Denoisers

Plug-and-play priors (PnP) is a methodology for regularized image reconstruction that specifies the prior through an image denoiser. While PnP algorithms are well understood for denoisers performing maximum a posteriori probability (MAP) estimation, they have not been analyzed for the minimum mean squared error (MMSE) denoisers. This letter addresses this gap by establishing the first theoretical convergence result for the iterative shrinkage/thresholding algorithm (ISTA) variant of PnP for MMSE denoisers. We show that the iterates produced by PnP-ISTA with an MMSE denoiser converge to a stationary point of some global cost function. We validate our analysis on sparse signal recovery in compressive sensing by comparing two types of denoisers, namely the exact MMSE denoiser and the approximate MMSE denoiser obtained by training a deep neural net.

An Event-Based Stealthy Attack on Remote State Estimation

Security issues in cyber-physical systems (CPSs) have gained increasing attention in recent years due to the importance and unavoidable vulnerability of CPSs. This article focuses on designing an intelligent online attack, which can compromise a sensor, eavesdrop measurements, and inject false feedback information, against remote state estimation. From the viewpoint of the attacker, we design an event-based attack strategy to degrade the estimation quality with an arbitrary communication rate stealthy constraint. The approximate minimum mean-squared error estimation algorithm from the viewpoint of the attacker is derived under a Gaussian assumption. Furthermore, the relation between the attack threshold and the scheduling threshold is obtained in a closed form. We show that the mean-squared stability condition of the estimation is weakened under the attack. Two examples are provided to demonstrate the main results.

Computationally Simple MMSE (A-Optimal) Adaptive Beam-Pattern Design for MIMO Active Sensing Systems via a Linear-Gaussian Approximation

This paper presents an approximate minimum mean squared error (MMSE) adaptive beam-pattern design (ABD) method for MIMO active sensing systems. The proposed approximate MMSE ABD method leverages the physics of the MIMO arrays to provide a linear-Gaussian approximation that is specific to MIMO active sensing systems, and yields a computationally simple optimization problem. Computational complexity analysis confirms this theoretical reduction in the number of floating-point operations required, most notably that evaluation of the proposed approximate optimization cost function grows polynomially with the number of targets being tracked, whereas for evaluation of the exact cost the growth is exponential. Additionally, numerical results indicate that, even for a simple scenario with a single target being tracked, the proposed approximate MMSE ABD method does indeed reduce the mean squared error of target parameter estimation compared to the nonadaptive case, with a reduction in computation time of four orders of magnitude compared to exact MMSE ABD.

Low complexity sparse Bayesian estimation for UWB radar imaging

Ultra-wideband (UWB) radar imaging can provide high-resolution images of obscured objects using radio-frequency signals. Due to its vast applications, UWB radar imaging received considerable attention in the past decade. Compressive sensing (CS) has been used as a viable solution for the larger data required by radar imaging. The advances of CS-based UWB-radar imaging is burdened by the complexity of the reconstruction algorithms and their weak noise immunity. Exploiting the structure of the basis-matrix, a low-complexity Bayesian-based estimation algorithm is proposed. The algorithm takes advantage of the radar-return statistic to find an approximate minimum mean-square error estimate of the radar image. The low complexity is achieved by utilising the block-matrix-inversion formula to execute the algorithm in an order-recursive manner. Further simplification is achieved by using exponential-sum formula to find the correlation between the columns of the basis-matrix. The proposed algorithm is evaluated over experimental and simulated data. The results show faster processing time compared to other known algorithms, with comparable reconstruction quality.

Particle filters for partially-observed Boolean dynamical systems

Partially-observed Boolean dynamical systems (POBDS) are a general class of nonlinear models with application in estimation and control of Boolean processes based on noisy and incomplete measurements. The optimal minimum mean square error (MMSE) algorithms for POBDS state estimation, namely, the Boolean Kalman filter (BKF) and Boolean Kalman smoother (BKS), are intractable in the case of large systems, due to computational and memory requirements. To address this, we introduce approximate MMSE filtering and smoothing algorithms based on the auxiliary particle filter (APF) method, which are called APF–BKF and APF–BKS, respectively. For joint state and parameter estimation, the APF–BKF is used jointly with maximum-likelihood (ML) methods for simultaneous state and parameter estimation in POBDS models. In the case the unknown parameters are discrete, the proposed ML adaptive filter consists of multiple APF–BKFs running in parallel, in a manner reminiscent of the Multiple Model Adaptive Estimation (MMAE) method in classical linear filtering theory. In the presence of continuous parameters, the proposed ML adaptive filter is based on an efficient particle-based expectation maximization (EM) algorithm for the POBDS model, which is based on a modified Forward Filter Backward Simulation (FFBSi) in combination with the APF–BKS. The performance of the proposed particle-based adaptive filters is assessed through numerical experiments using a POBDS model of the well-known cell cycle gene regulatory network observed through noisy RNA-Seq time series data.

Approximate MMSE Estimator for Linear Dynamic Systems With Gaussian Mixture Noise

In this work, we propose an approximate minimum mean-square error filter for linear dynamic systems with Gaussian Mixture (GM) noise. The proposed estimator tracks each component of the GM posterior with an individual filter and minimizes the trace of the covariance matrix of the bank of filters, as opposed to minimizing the MSE of individual filters filters in the commonly used Gaussian sum filter (GSF). Hence, the spread of means in the proposed method is smaller than that of GSF which makes it more robust to removing components. Consequently, reduction schemes with lower computational complexity can be used with the proposed filter without losing estimation accuracy and precision. This is supported through simulations on synthetic data as well as experimental data related to an indoor localization system. Additionally, we show that in two limit cases the state estimation provided by our proposed method converges to that of GSF, and we provide simulation results supporting this in other cases.

Maximum-Likelihood Adaptive Filter for Partially Observed Boolean Dynamical Systems

Partially-observed Boolean dynamical systems (POBDS) are a general class of nonlinear models with application in estimation and control of Boolean processes based on noisy and incomplete measurements. The optimal minimum mean square error (MMSE) algorithms for POBDS state estimation, namely, the Boolean Kalman filter (BKF) and Boolean Kalman smoother (BKS), are intractable in the case of large systems, due to computational and memory requirements. To address this, we propose approximate MMSE filtering and smoothing algorithms based on the auxiliary particle filter (APF) method from sequential Monte-Carlo theory. These algorithms are used jointly with maximum-likelihood (ML) methods for simultaneous state and parameter estimation in POBDS models. In the presence of continuous parameters, ML estimation is performed using the expectation-maximization (EM) algorithm; we develop for this purpose a special smoother which reduces the computational complexity of the EM algorithm. The resulting particle-based adaptive filter is applied to a POBDS model of Boolean gene regulatory networks observed through noisy RNA-Seq time series data, and performance is assessed through a series of numerical experiments using the well-known cell cycle gene regulatory model.

High-Throughput Data Detection for Massive MU-MIMO-OFDM Using Coordinate Descent

Data detection in massive multi-user (MU) multiple-input multiple-output (MIMO) wireless systems is among the most critical tasks due to the excessively high implementation complexity. In this paper, we propose a novel, equalization-based soft-output data-detection algorithm and corresponding reference FPGA designs for wideband massive MU-MIMO systems that use orthogonal frequency-division multiplexing (OFDM). Our data-detection algorithm performs approximate minimum mean-square error (MMSE) or box-constrained equalization using coordinate descent. We deploy a variety of algorithm-level optimizations that enable near-optimal error-rate performance at low implementation complexity, even for systems with hundreds of base-station (BS) antennas and thousands of subcarriers. We design a parallel VLSI architecture that uses pipeline interleaving and can be parametrized at design time to support various antenna configurations. We develop reference FPGA designs for massive MU-MIMO-OFDM systems and provide an extensive comparison to existing designs in terms of implementation complexity, throughput, and error-rate performance. For a 128 BS antenna, 8-user massive MU-MIMO-OFDM system, our FPGA design outperforms the next-best implementation by more than $2.6 \times $ in terms of throughput per FPGA look-up tables.

On Set-Valued Kalman Filtering and Its Application to Event-Based State Estimation

Motivated by challenges in state estimation with event-based measurement updates, the properties of the exact and approximate set-valued Kalman filters with multiple sensor measurements for linear time-invariant systems are investigated in this work. First, we show that the exact and the proposed approximate set-valued filters are independent of the fusion sequence at each time instant. Second, the boundedness of the size of the set of estimation means is proved for the exact set-valued filter. For the approximate set-valued filter, if the closed-loop matrix is contractive, then the set of estimation means has a bounded size asymptotically; otherwise a nonsingular linear transform is constructed such that the size of the set of estimation means for the transformed states is asymptotically bounded. Third, the effect of set-valued measurements on the size of the set of estimation means is analyzed and conditions for performance improvement in terms of smaller size of the set of estimation means are proposed. Finally, the results are applied to event-based estimation, which allow the event-triggering conditions to be designed by considering requirements on performance and communication rates. The efficiency of the proposed results are illustrated by simulation examples and comparison with the approximate event-based MMSE estimator and the Kalman filter with intermittent observations.

Active Classification for POMDPs: A Kalman-Like State Estimator

The problem of state tracking with active observation control is considered for a system modeled by a discrete-time, finite-state Markov chain observed through conditionally Gaussian measurement vectors. The measurement model statistics are shaped by the underlying state and an exogenous control input, which influence the observations' quality. Exploiting an innovations approach, an approximate minimum mean-squared error (MMSE) filter is derived to estimate the Markov chain system state. To optimize the control strategy, the associated mean-squared error is used as an optimization criterion in a partially observable Markov decision process formulation. A stochastic dynamic programming algorithm is proposed to solve for the optimal solution. To enhance the quality of system state estimates, approximate MMSE smoothing estimators are also derived. Finally, the performance of the proposed framework is illustrated on the problem of physical activity detection in wireless body sensing networks. The power of the proposed framework lies within its ability to accommodate a broad spectrum of active classification applications including sensor management for object classification and tracking, estimation of sparse signals and radar scheduling.

Sparse Reconstruction Using Distribution Agnostic Bayesian Matching Pursuit

A fast matching pursuit method using a Bayesian approach is introduced for sparse signal recovery. This method performs Bayesian estimates of sparse signals even when the signal prior is non-Gaussian or unknown. It is agnostic on signal statistics and utilizes a priori statistics of additive noise and the sparsity rate of the signal, which are shown to be easily estimated from data if not available. The method utilizes a greedy approach and order-recursive updates of its metrics to find the most dominant sparse supports to determine the approximate minimum mean-square error (MMSE) estimate of the sparse signal. Simulation results demonstrate the power and robustness of our proposed estimator.

Bayesian Estimation With Distance Bounds

We consider the problem of estimating a random state vector when there is information about the maximum distances between its subvectors. The estimation problem is posed in a Bayesian framework in which the minimum mean square error (MMSE) estimate of the state is given by the conditional mean. Since finding the conditional mean requires multidimensional integration, an approximate MMSE estimator is proposed. The performance of the proposed estimator is evaluated in a positioning problem. Finally, the application of the estimator in inequality constrained recursive filtering is illustrated by applying the estimator to a dead-reckoning problem. The MSE of the estimator is compared with two related posterior Cram\'er-Rao bounds.

Low-Complexity Soft-Decision Feedback Turbo Equalization for MIMO Systems With Multilevel Modulations

Many communication systems today encounter the problem of data transmission over a channel with intersymbol interference (ISI). The purpose of this paper is to develop a low-complexity iterative soft-decision feedback equalization (SDFE) receiver for severe frequency-selective ISI channels in multiple-input-multiple-output (MIMO) communication systems. The proposed SDFE algorithm offers a novel approach to combat error propagation. In addition, its computational complexity only linearly grows with the number of equalizer coefficients, compared with the quadratic complexity of minimum mean square error (MMSE)-based linear turbo equalizer with time-varying coefficients. The performance of the proposed detection scheme is verified through simulations using different signal constellations. In addition, convergence properties of the proposed SDFE algorithm are also analyzed using an extrinsic information transfer (EXIT) chart and verified by simulations in a severe ISI channel. Simulation results show that our proposed algorithm has significant improvement over the Approximate MMSE linear turbo equalizer proposed by Tüchler et al. Moreover, we show that the performance of the proposed equalization scheme significantly improves when higher order constellations are used for digital modulation.

Blind CFO Estimation for Linearly Precoded OFDMA Uplink

This paper presents a novel method for blind carrier-frequency-offset (CFO) estimation in linearly precoded OFDMA uplink. Our investigation starts from the single-user equivalent (SUE) scenario presented in where active users in the network are categorized into a number of reference (synchronized) users (RUs) and a new (asynchronous) user (NU). The major idea is to take advantage of time correlation induced by the linear precoder, which offers a second-order moments-based blind CFO estimation. The precoder design is carefully performed in terms of CFO identifiability, estimation accuracy and overall system performance. In the multiuser scenario, where all users can be misaligned in frequency domain, the proposed CFO estimator is capable of mitigating a considerable portion of interference from neighboring users through exploitation of a novel time-frequency multiuser data-mapping (MU-DM) scheme. To demonstrate the multiuser interference (MUI)-resilience feature of proposed scheme, theoretical analysis is performed through derivation of approximate minimum-mean-square error (MMSE) in both SUE and multiuser scenarios. It is shown that by exploitation of the proposed MU-DM scheme, the approximate multiuser MMSE is very close to that of SUE scenario. Simulation results show that the proposed approach outperforms state-of-the-art approaches in both SUE and multiuser scenarios.

Optimizations of a MIMO Relay Network

Relay networks have received considerable attention recently, especially when limited size and power resources impose constraints on the number of antennas within a wireless sensor network. In this context, signal processing techniques play a fundamental role, and optimality within a given relay architecture can be achieved under several design criteria. In this paper, we extend recent optimal minimum-mean-square-error (MMSE) and signal-to-noise ratio (SNR) designs of relay networks to the corresponding multiple-input-multiple-output (MIMO) scenarios, whereby the source, relays and destination comprise multiple antennas. We investigate maximum SNR solutions subject to power constraints and zero-forcing (ZF) criteria, as well as approximate MMSE equalizers with specified target SNR and power constraint at the receiver. We also maximize the transmission rate between the source and destination subject to power constraint at the receiver.