Maximum Likelihood Solution Research Articles

Localization of Multiple Sources in the Presence of Model Errors by Total Least Squares

We present a novel solution to the localization of multiple sources by a passive sensor array, based on the total-least-squares criterion, which is aimed at coping with the critical problem of model errors. Simulation results, demonstrating the performance superiority over the deterministic maximum likelihood solution, especially in the case of highly correlated sources, are included.

A semi-analytical solution to the maximum-likelihood fit of Poisson data to a linear model using the Cash statistic

The Cash statistic, also known as the statistic, is commonly used for the analysis of low-count Poisson data, including data with null counts for certain values of the independent variable. The use of this statistic is especially attractive for low-count data that cannot be combined, or re-binned, without loss of resolution. This paper presents a new maximum-likelihood solution for the best-fit parameters of a linear model using the Poisson-based Cash statistic. The solution presented in this paper provides a new and simple method to measure the best-fit parameters of a linear model for any Poisson-based data, including data with null counts. In particular, the method enforces the requirement that the best-fit linear model be non-negative throughout the support of the independent variable. The method is summarized in a simple algorithm to fit Poisson counting data of any size and counting rate with a linear model, by-passing entirely the use of the traditional statistic.

Localization of multiple sources with known waveforms by array response matching

We present a novel and computationally-efficient solution to the localization of multiple sources with known waveforms, based on array response matching. We first derive the multidimensional array response matching (ARM) solution, which amounts to a minimization problem involving the location of the sources. To simplify the computational complexity, we derive a 1-dimensional version, which is closely related to the MVDR solution. We then introduce a linearly constrained version of this solution and apply it to exploit a-priori knowledge regarding the waveforms of the sources. The proposed solution involves only a 1-dimensional search and is therefore computationally much simpler than the corresponding maximum likelihood (ML) solution, which involves an iterative multidimensional search. Simulation results, demonstrating the close similarity in performance of the proposed solution and the ML solution, are included.

Maximum likelihood-based robust state estimation over a horizon length during measurement outliers

In this paper, a novel approach towards horizon-based maximum likelihood (ML) state estimator is proposed that makes the state estimation process more robust against unmodeled and unstructured noise and disturbances in the state-space models. State space models provide a powerful way to perform state and parameter estimation for dynamical systems. However, if the measurements are contaminated by outliers and disturbances with no known models, the estimation process is highly biased. A ML-based approach is used to find a batch solution for the filtering problem. Based on ML solution, a robust algorithm is proposed that seamlessly estimates dynamic state in the presence of zero or non-zero mean measurement outliers. The proposed algorithm that dictates this switching uses an explicit outlier detection mechanism that enables its seamless working. Simulations have been carried out for a moving target tracking application as an example to demonstrate the resilience of the proposed method against zero-mean and non-zero mean measurement outliers in comparison to state-of-the-art methods.

Imaging the 511 keV Positron Annihilation Sky with COSI

Abstract The balloon-borne Compton Spectrometer and Imager (COSI) had a successful 46-day flight in 2016. The instrument is sensitive to photons in the energy range 0.2–5 MeV. Compton telescopes have the advantage of a unique imaging response and provide the possibility of strong background suppression. With its high-purity germanium detectors, COSI can precisely map γ-ray line emission. The strongest persistent and diffuse γ-ray line signal is the 511 keV emission line from the annihilation of electrons with positrons from the direction of the Galactic center. While many sources have been proposed to explain the amount of positrons, , the true contributions remain unsolved. In this study, we aim at imaging the 511 keV sky with COSI and pursue a full-forward modeling approach, using a simulated and binned imaging response. For the strong instrumental background, we describe an empirical approach to take the balloon environment into account. We perform two alternative methods to describe the signal: Richardson–Lucy deconvolution, an iterative method toward the maximum likelihood solution, and model fitting with predefined emission templates. Consistently with both methods, we find a 511 keV bulge signal with a flux between 0.9 and , confirming earlier measurements, and also indications of more extended emission. The upper limit we find for the 511 keV disk, , is consistent with previous detections. For large-scale emission with weak gradients, coded aperture mask instruments suffer from their inability to distinguish isotropic emission from instrumental background, while Compton telescopes provide a clear imaging response, independent of the true emission.

Maximum likelihood low-complexity GSM detection for large MIMO systems

Hard-Output Maximum Likelihood (ML) detection for Generalized Spatial Modulation (GSM) systems involves obtaining the ML solution of a number of different MIMO subproblems, with as many possible antenna configurations as subproblems. Obtaining the ML solution of all of the subproblems has a large computational complexity, especially for large GSM MIMO systems. In this paper, we present two techniques for reducing the computational complexity of GSM ML detection.The first technique is based on computing a box optimization bound for each subproblem. This, together with sequential processing of the subproblems, allows fast discarding of many of these subproblems. The second technique is to use a Sphere Detector that is based on box optimization for the solution of the subproblems. This Sphere Detector reduces the number of partial solutions explored in each subproblem. The experiments show that these techniques are very effective in reducing the computational complexity in large MIMO setups.

Understanding GANs in the LQG Setting: Formulation, Generalization and Stability

Generative Adversarial Networks (GANs) have become a popular method to learn a probability model from data. In this paper, we provide an understanding of basic issues surrounding GANs including their formulation, generalization and stability on a simple LQG benchmark where the generator is Linear , the discriminator is Quadratic and the data has a high-dimensional Gaussian distribution. Even in this simple benchmark, the GAN problem has not been well-understood as we observe that existing state-of-the-art GAN architectures may fail to learn a proper generative distribution owing to (1) stability issues (i.e., convergence to bad local solutions or not converging at all), (2) approximation issues (i.e., having improper global GAN optimizers caused by inappropriate GAN’s loss functions), and (3) generalizability issues (i.e., requiring large number of samples for training). In this setup, we propose a GAN architecture which recovers the maximum-likelihood solution and demonstrates fast generalization. Moreover, we analyze global stability of different computational approaches for the proposed GAN and highlight their pros and cons. Finally, through experiments on MNIST and CIFAR-10 datasets, we outline extensions of our model-based approach to design GANs in more complex setups than the considered Gaussian benchmark.

A latent variable model for two-dimensional canonical correlation analysis and the variational inference

Describing the dimension reduction (DR) techniques by means of probabilistic models has recently been given special attention. Probabilistic models, in addition to a better interpretability of the DR methods, provide a framework for further extensions of such algorithms. One of the new approaches to the probabilistic DR methods is to preserving the internal structure of data. It is meant that it is not necessary that the data first be converted from the matrix or tensor format to the vector format in the process of dimensionality reduction. In this paper, a latent variable model for matrix-variate data for canonical correlation analysis (CCA) is proposed. Since in general there is not any analytical maximum likelihood solution for this model, we present two approaches for learning the parameters. The proposed methods are evaluated using the synthetic data in terms of convergence and quality of mappings. Also, real data set is employed for assessing the proposed methods with several probabilistic and none-probabilistic CCA based approaches. The results confirm the superiority of the proposed methods with respect to the competing algorithms. Moreover, this model can be considered as a framework for further extensions.

Joint Estimation of RF Impairments, Channel, and Low Complexity Iterative Equalization Technique for High Mobility SC-FDMA/OFDMA Uplink Systems

In this paper, we consider the problem of joint estimation and equalization of doubly selective channels and all radio frequency impairments, namely transmitter IQ imbalance (TIQI), receiver IQI (RIQI) and carrier frequency offsets (CFO) in the uplink of single carrier frequency division multiple access (SC-FDMA) and orthogonal frequency division multiple access (OFDMA) systems with direct conversion transceiver architecture and high mobility users. We combine the effect of TIQI, RIQI and DSC to a new effective channel and propose the use of a Hermite polynomial based basis expansion model for approximating it to avoid the identifiability issue associated with this complete parameter estimation problem. As the maximum likelihood solution to this joint estimation problem requires complex multi-dimensional searches, we propose a novel subspace nulling based iterative estimation technique that effectively reduces the multi-user interference and replaces the multi-dimensional search with single-dimensional searches. Further, without much compromise on the performance, we also derive a low complexity version of the proposed technique. Additionally, we derive the Cramer-Rao bound for the joint estimation of CFO and effective channel. Moreover, we propose two equalization techniques for iteratively equalizing the mirror subcarrier pairs. The proposed techniques do not put any restriction on the carrier assignment scheme and provide good performance even at high mobile speeds.

Maximum likelihood solutions for multicriterial choice problems

We introduce the notion of a maximum likelihood optimal decision alternative for the choice problem with a finite set of decision alternatives, assuming a general parametric preference model of the decision maker. We also develop an optimization-based method for the identification of such alternative for the cases in which the parametric preference model is based on uncertain intervals for criterion trade-offs. The suggested approach can be seen as generalising stochastic multicriteria acceptability analysis (SMAA) to a wider modelling setting. It also provides a maximum likelihood interpretation of the SMAA acceptability index.

Bayesian weighted inference from surveys

SummaryData from large surveys are often supplemented with sampling weights that are designed to reflect unequal probabilities of response and selection inherent in complex survey sampling methods. We propose two methods for Bayesian estimation of parametric models in a setting where the survey data and the weights are available, but where information on how the weights were constructed is unavailable. The first approach is to simply replace the likelihood with the pseudo likelihood in the formulation of Bayes theorem. This is proven to lead to a consistent estimator but also leads to credible intervals that suffer from systematic undercoverage. Our second approach involves using the weights to generate a representative sample which is integrated into a Markov chain Monte Carlo (MCMC) or other simulation algorithms designed to estimate the parameters of the model. In the extensive simulation studies, the latter methodology is shown to achieve performance comparable to the standard frequentist solution of pseudo maximum likelihood, with the added advantage of being applicable to models that require inference via MCMC. The methodology is demonstrated further by fitting a mixture of gamma densities to a sample of Australian household income.

Fast Super-Paramagnetic Clustering

We map stock market interactions to spin models to recover their hierarchical structure using a simulated annealing based Super-Paramagnetic Clustering (SPC) algorithm. This is directly compared to a modified implementation of a maximum likelihood approach we call fast Super-Paramagnetic Clustering (f-SPC). The methods are first applied to standard toy test-case problems, and then to a data-set of 447 stocks traded on the New York Stock Exchange (NYSE) over 1249 days. The signal to noise ratio of stock market correlation matrices is briefly considered. Our result recover approximately clusters representative of standard economic sectors and mixed ones whose dynamics shine light on the adaptive nature of financial markets and raise concerns relating to the effectiveness of industry based static financial market classification in the world of real-time data analytics. A key result is that we show that f-SPC maximum likelihood solutions converge to ones found within the Super-Paramagnetic Phase where the entropy is maximum, and those solutions are qualitatively better for high dimensionality data-sets.

Using Generalized Gaussian Distributions to Improve Regression Error Modeling for Deep Learning-Based Speech Enhancement

From a statistical perspective, the conventional minimum mean squared error (MMSE) criterion can be considered as the maximum likelihood (ML) solution under an assumed homoscedastic Gaussian error model. However, in this paper, a statistical analysis reveals the super-Gaussian and heteroscedastic properties of the prediction errors in nonlinear regression deep neural network (DNN)-based speech enhancement when estimating clean log-power spectral (LPS) components at DNN outputs with noisy LPS features in DNN input vectors. Accordingly, we propose treating all dimensions of the prediction error vector as statistically independent random variables and model them with generalized Gaussian distributions (GGDs). Then, the objective function with the GGD error model is derived according to the ML criterion. Experiments on the TIMIT corpus corrupted by simulated additive noises show consistent improvements of our proposed DNN framework over the conventional DNN framework in terms of various objective quality measures under 14 unseen noise types evaluated and at various signal-to-noise ratio levels. Furthermore, the ML optimization objective with GGD outperforms the conventional MMSE criterion, achieving improved generalization and robustness.

Linear regression with bivariate response variable containing missing data. Strategies to increase prediction precision

In this study we focus on prediction precision for linear regression with a bivariate response variable, where the response variable of primary interest contains missing data in the training data set. We derive and provide the maximum likelihood solution and a Bayesian method based on a conjugate prior distribution. In particular we evaluate strategies in how to “borrow prediction strength” from the full set of data to prediction associated with the missing data. Regularization of the maximum likelihood estimator is theoretically shown to be beneficial, and we derive methods for how to implement such regularization under frequentist and Bayesian inference, including available software as a R-package.

Optimum Low-Complexity Decoder for Spatial Modulation

In this paper, a novel low-complexity detection algorithm for spatial modulation (SM), referred to as the minimum-distance of maximum-length (m-M) algorithm, is proposed and analyzed. The proposed m-M algorithm is a smart searching method that is applied for the SM tree-search decoders. The behavior of the m-M algorithm is studied for three different scenarios: i) perfect channel state information at the receiver side (CSIR), ii) imperfect CSIR of a fixed channel estimation error variance, and iii) imperfect CSIR of a variable channel estimation error variance. Moreover, the complexity of the m-M algorithm is considered as a random variable, which is carefully analyzed for all scenarios, using probabilistic tools. Based on a combination of the sphere decoder (SD) and ordering concepts, the m-M algorithm guarantees to find the maximum-likelihood (ML) solution with a significant reduction in the decoding complexity compared to SM-ML and existing SM-SD algorithms; it can reduce the complexity up to 94% and 85% in the perfect CSIR and the worst scenario of imperfect CSIR, respectively, compared to the SM-ML decoder. Monte Carlo simulation results are provided to support our findings as well as the derived analytical complexity reduction expressions.

Millimeter-Wave Downlink Positioning With a Single-Antenna Receiver

This paper addresses the problem of determining the unknown position of a mobile station for a mmWave multiple-input single-output (MISO) system. This setup is motivated by the fact that massive arrays will be initially implemented only on 5G base stations, likely leaving mobile stations with one antenna. The maximum likelihood solution to this problem is devised based on the time of flight and angle of departure of received downlink signals. While positioning in the uplink would rely on angle of arrival, it presents scalability limitations that are avoided in the downlink. To circumvent the multidimensional optimization of the optimal joint estimator, we propose two novel approaches amenable to practical implementation thanks to their reduced complexity. A thorough analysis, which includes the derivation of relevant Cramer–Rao lower bounds, shows that it is possible to achieve quasi-optimal performance even in presence of few transmissions, low signal-to-noise ratio (SNRs), and multipath propagation effects.

Identification of multiple leaks in pipeline III: Experimental results

Abstract Identifying multiple leaks in a water supply pipe is challenging because pipe systems are complex and many parameters need to be estimated. In order to solve this problem, in Part I and Part II of the series of papers Wang and Ghidaoui (2018, 2019), a linearized model of transient wave in pipes was introduced, a maximum likelihood (ML) scheme to estimate leaks was proposed (for large leak number, the iterative beamforming method was used to simplify the ML solution), and the model selection methods were used to decide the leak number. In the present paper (Part III), these methods are experimentally justified via transient tests from a newly-built high-density polyethylene (HDPE) pipe system in the Water Resources Research Laboratory at the Hong Kong University of Science and Technology. Since the experiments are conducted in a viscoelastic pipe, the previous transient wave model (for elastic pipes) is modified to quantify the viscoelastic behavior of pipe wall. Experimental results with two and three leaks show that the proposed approaches are able to accurately localize the multiple leaks and decide the number of leaks.

Categorical latent variable modeling utilizing fuzzy clustering generalized structured component analysis as an alternative to latent class analysis

Latent class analysis is becoming popular in many areas of education, psychology, social and behavioral sciences, public health, and medicine. However, it often suffers from identification issues due to the large number of parameters involved when using maximum likelihood (ML) estimation. Increasing the sample size, reducing sparseness, and strengthening the relationship between the observed variables and the latent variables all improve the information and thus reduce the identification issues, but the identification issue still affects the validity of parameter estimates in ML estimation and the definition of identification is not sufficient to guarantee the existence of an ML solution. In this paper, generalized structured component analysis (GSCA), which is a component-based approach that utilizes optimal scaling and fuzzy clustering, is applied to avoid these identification issues and develop more stable solutions for the heterogeneity of a population based on a set of categorical responses. Testing our proposed new approach, component-based (CB) latent class analysis (LCA), on real world substance use data from Add Health produced not only the same features as those yielded by conventional ML LCA but also stable estimation without identification issues. Comparing the results obtained from ML LCA using Mplus and poLCA in R, with those from our proposed CB LCA using GSCA in R revealed a similar number of latent classes and posterior probabilities and only minor discrepancies in individual latent class classifications when the posterior probabilities of membership are not distinct.

A probabilistic method for the estimation of earthquake source parameters from spectral inversion: application to the 2016–2017 Central Italy seismic sequence

We develop a probabilistic framework based on the conjunction of states of information between data and model, to jointly retrieve earthquake source parameters and anelastic attenuation factor from inversion of displacement amplitude spectra. The evaluation of the joint probability density functions (PDFs) enables us to take into account between-parameter correlations in the final estimates of the parameters and related uncertainties. Following this approach, we first search for the maximum of the a-posteriori PDF through the basin hopping technique that couples a global exploration built on a Markov chain with a local deterministic maximization. Then we compute statistical indicators (mean, variance and correlation coefficients) on source parameters and anelastic attenuation through integration of the PDF in the vicinity of the maximum likelihood solution. Definition of quality criteria based on the signal to noise ratio and the similarity of the marginal PDFs with a Gaussian function enable us to define the frequency domain for the inversion and to get rid of unconstrained solutions. We perform synthetic tests to assess theoretical correlations as a function of the signal to noise ratio and to define the minimum bandwidth around the corner frequency for consistent parameter resolution. As an application, we finally estimate the source parameters for the 2016-2017 Central Italy seismic sequence. We found that the classical scaling between the seismic moment and the corner frequency holds, with an average stress drop of $\Delta\sigma$ = 2.1 +- 0.3 MPa. However, the main events in the sequence exhibit a stress drop larger than the average value. Finally, the small seismic efficiency indicates a stress overshoot, possibly due to dynamic effects or large frictional efficiency.

Low-Rank Structured Covariance Matrix Estimation

The covariance matrix estimation problem is posed in both the Bayesian and frequentist settings as the solution of a maximum a posteriori (MAP) or maximum likelihood (ML) optimization, respectively, when the true covariance consists of a known (or bounded) noise floor and a low-rank component. Persymmetric structure may also be assumed. The MAP and ML solutions with the non-convex rank constraint are shown to be a simple scalar thresholding of eigenvalues of a suitably translated and projected sample covariance matrix. No iterative optimization is required; therefore, the computation is suited to real-time applications. Our proof is short and elementary without resorting to the duality theory. Numerical results are presented to illustrate the improved estimation performance obtained by incorporating the structural constraints on the unknown covariance.