Stochastic Maximum Likelihood Research Articles

Expectation-maximization type algorithms for direction of arrival estimation in unknown nonuniform noise

The expectation–maximization (EM) algorithm updates all of the parameter estimates simultaneously, which is not applicable to direction of arrival (DOA) estimation in unknown nonuniform noise. In this work, we present several computationally efficient EM-type algorithms, which update the parameter estimates sequentially, for solving both the deterministic and stochastic maximum–likelihood (ML) direction finding problems in unknown nonuniform noise. Specifically, we design a generalized EM (GEM) algorithm and a space-alternating generalized EM (SAGE) algorithm for computing the deterministic ML estimator. Simulation results show that the SAGE algorithm outperforms the GEM algorithm. Moreover, we design two SAGE algorithms for computing the stochastic ML estimator, in which the first updates the DOA estimates simultaneously while the second updates the DOA estimates sequentially. Simulation results show that the second SAGE algorithm outperforms the first one.

Stochastic Maximum Likelihood Direction Finding in the Presence of Nonuniform Noise Fields

The maximum likelihood (ML) technique plays an important role in direction-of-arrival (DOA) estimation. In this paper, we employ and design the expectation–conditional maximization either (ECME) algorithm, a generalization of the expectation–maximization algorithm, for solving the ML direction finding problem of stochastic sources, which may be correlated, in unknown nonuniform noise. Unlike alternating maximization, the ECME algorithm updates both the source and noise covariance matrix estimates by explicit formulas, and can guarantee that both estimates are positive semi-definite and definite, respectively. Thus, the ECME algorithm is computationally efficient and operationally stable. Simulation results confirm that the ECME algorithm can efficiently obtain the ML based DOA estimate of each stochastic source.

A Robust and Statistically Efficient Maximum-Likelihood Method for DOA Estimation Using Sparse Linear Arrays

A recent trend of research on direction-of-arrival (DOA) estimation is to localize more uncorrelated sources than sensors by using a proper sparse linear array (SLA), at the cost of robustness to source correlations even in the regime of less sources than sensors. This paper is devoted to proposing one algorithm that can simultaneously tackle two challenging scenarios: 1) more uncorrelated sources than sensors and 2) highly correlated or coherent sources. In order to statistically efficiently localize a maximal number of uncorrelated sources, we use the stochastic maximum likelihood (SML) criterion and propose an effective algorithm based on elegant problem reformulations and the alternating direction method of multipliers (ADMM). Moreover, we prove that the SML is robust to source correlations under mild conditions though it is derived under the assumption of uncorrelated sources. The proposed algorithm is usable for arbitrary SLAs (e.g., minimum redundancy arrays, nested arrays and coprime arrays) and is named as <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">m</i> aximum-likelihood <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">e</i> stimation via <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">s</i> equential <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">A</i> DMM (MESA). Extensive numerical results are provided that collaborate our analysis and demonstrate the statistical efficiency and robustness of MESA against state-of-the-art algorithms. Our results also imply that it is possible to localize more sources than sensors in presence of high source correlations.

Maximum Likelihood-Based Gridless DoA Estimation Using Structured Covariance Matrix Recovery and SBL With Grid Refinement

We consider the parametric data model employed in applications such as line spectral estimation and direction-of-arrival estimation. We focus on the stochastic maximum likelihood estimation (MLE) framework and offer approaches to estimate the parameter of interest in a gridless manner, overcoming the model complexities of the past. This progress is enabled by the modern trend of reparameterization of the objective and exploiting the sparse Bayesian learning (SBL) approach. The latter is shown to be a correlation-aware method, and for the underlying problem it is identified as a grid-based technique for recovering a structured covariance matrix of the measurements. For the case when the structured matrix is expressible as a sampled Toeplitz matrix, such as when measurements are sampled in time or space at regular intervals, additional constraints and reparameterization of the SBL objective leads to the proposed structured matrix recovery technique based on MLE. The proposed optimization problem is non-convex, and we propose a majorization-minimization based iterative procedure to estimate the structured matrix; each iteration solves a semidefinite program. We recover the parameter of interest in a gridless manner by appealing to the Caratheodory-Fejer result on decomposition of PSD Toeplitz matrices. For the general case of irregularly spaced time or spatial samples, we propose an iterative SBL procedure that refines grid points to increase resolution near potential source locations, while maintaining a low per iteration complexity. We provide numerical results to evaluate and compare the performance of the proposed techniques with other gridless techniques, and the CRB. The proposed correlation-aware approach is more robust to environmental/system effects such as low number of snapshots, correlated sources, small separation between source locations and improves sources identifiability.

Efficient computation of ML DOA estimates under unknown nonuniform sensor noise powers

This paper presents an efficient method for computing Maximum Likelihood (ML) direction-of-arrival (DOA) estimates in scenarios in which the sensor noise powers are nonuniform and unknown. The method combines the Alternating Projection (AP) algorithm for coarsely locating additional DOAs and Newton iterations for finally obtaining the ML estimates. Compared with the existing approaches, the method reduces the computational burden significantly due to the small number of Newton iterations required and to the efficient computation of each iteration. Specifically, the iterations are computed in a small number of arithmetic operations thanks to the closed-form formulas for the gradient and Hessian of the ML cost functions presented in this paper. The method’s total computational burden is of just a few mega-flops (mega floating-point operations) in typical cases. We present the method for the deterministic and stochastic ML estimators. Then, an analysis of the deterministic ML cost function’s gradient reveals an unexpected drawback: its associated estimator is either degenerate or inconsistent. Finally, we assess the method’s root-mean-square (RMS) error and computational burden numerically and compare it with other relevant estimators in the literature.

DrugnomeAI is an ensemble machine-learning framework for predicting druggability of candidate drug targets

The druggability of targets is a crucial consideration in drug target selection. Here, we adopt a stochastic semi-supervised ML framework to develop DrugnomeAI, which estimates the druggability likelihood for every protein-coding gene in the human exome. DrugnomeAI integrates gene-level properties from 15 sources resulting in 324 features. The tool generates exome-wide predictions based on labelled sets of known drug targets (median AUC: 0.97), highlighting features from protein-protein interaction networks as top predictors. DrugnomeAI provides generic as well as specialised models stratified by disease type or drug therapeutic modality. The top-ranking DrugnomeAI genes were significantly enriched for genes previously selected for clinical development programs (p value < 1 × 10−308) and for genes achieving genome-wide significance in phenome-wide association studies of 450 K UK Biobank exomes for binary (p value = 1.7 × 10−5) and quantitative traits (p value = 1.6 × 10−7). We accompany our method with a web application (http://drugnomeai.public.cgr.astrazeneca.com) to visualise the druggability predictions and the key features that define gene druggability, per disease type and modality.

Critical Review of Basic Methods on DoA Estimation of EM Waves Impinging a Spherical Antenna Array

Direction-of-arrival (DoA) estimation of electromagnetic (EM) waves impinging on a spherical antenna array in short time windows is examined in this paper. Reflected EM signals due to non-line-of-sight propagation measured with a spherical antenna array can be coherent and/or highly correlated in a snapshot. This makes spectral-based methods inefficient. Spectral methods, such as maximum likelihood (ML) methods, multiple signal classification (MUSIC), and beamforming methods, are theoretically and systematically investigated in this study. MUSIC is an approach used for frequency estimation and radio direction finding, ML is a technique used for estimating the parameters of an assumed probability distribution for given observed data, and PWD applies a Fourier transform to the capture response and produces them in the frequency domain. Although they have been previously adapted and used to estimate DoA of EM signals impinging on linear and planar antenna array configurations, this paper investigates their suitability and effectiveness for a spherical antenna array. Various computer simulations were conducted, and plots of root-mean-square error (RMSE) against the square root of the Cramér–Rao lower bound (CRLB) were generated and used to evaluate the performance of each method. Numerical experiments and results from measured data show the degree of appropriateness and efficiency of each method. For instance, the techniques exhibit identical performance to that in the wideband scenario when the frequency f = 8 GHz, f = 16 GHz, and f = 32 GHz, but f = 16 GHz performs best. This indicates that the difference between the covariance matrix of the signal is coherent and that the steering vectors of signals impinging from that angle are small. MUSIC and PWD share the same problems in the single-frequency scenario as in the wideband scenario when the delay sample d = 0. Consequently, the DoA estimation obtained with ML techniques is more suitable, less biased, and more robust against noise than beamforming and MUSIC techniques. In addition, deterministic ML (DML) and weighted subspace fitting (WSF) techniques show better DoA estimation performance than the stochastic ML (SML) technique. For a large number of snapshots, WSF is a better choice because it is more computationally efficient than DML. Finally, the results obtained indicate that WSF and ML methods perform better than MUSIC and PWD for the coherent or partially correlated signals studied.

DOA Estimation in Heteroscedastic Noise with sparse Bayesian Learning

We consider direction of arrival (DOA) estimation from long-term observations in a noisy environment. In such an environment the noise source might evolve, causing the stationary models to fail. Therefore a heteroscedastic Gaussian noise model is introduced where the variance can vary across observations and sensors. The source amplitudes are assumed independent zero-mean complex Gaussian distributed with unknown variances (i.e., source powers), leading to stochastic maximum likelihood (ML) DOA estimation. The DOAs are estimated from multi-snapshot array data using sparse Bayesian learning (SBL) where the noise is estimated across both sensors and snapshots.

On DoA estimation for rotating arrays using stochastic maximum likelihood approach

The flexibility needed to construct DoA estimators that can be used with rotating arrays subject to rapid variations of the signal frequency is offered by the stochastic maximum likelihood approach. Using a combination of analytic methods and Monte Carlo simulations, we show that for low and moderate source correlations the stochastic maximum likelihood estimator that assumes noncorrelated sources has accuracy comparable to the estimator that includes the correlation coefficient as one of the parameters. We propose several fast approximations of the stochastic maximum likelihood estimator and compare their accuracy with the Crámer-Rao lower bound. We also discuss the model order selection problem for the binary- and multiple-hypotheses cases.

General Improvements of Heuristic Algorithms for Low Complexity DOA Estimation

Heuristic algorithms are considered to be effective approaches for super-resolution DOA estimations such as Deterministic Maximum Likelihood (DML), Stochastic Maximum Likelihood (SML), and Weighted Subspace Fitting (WSF) which are involved in nonlinear multi-dimensional optimization. Traditional heuristic algorithms usually need a large number of particles and iteration times. As a result, the computational complexity is still a bit high, which prevents the application of these super-resolution techniques in real systems. To reduce the computational complexity of heuristic algorithms for these super-resolution techniques of DOA, this paper proposes three general improvements of heuristic algorithms, i.e., the optimization of the initialization space, the optimization of evolutionary strategies, and the usage of parallel computing techniques. Simulation results show that the computational complexity can be greatly reduced while these improvements are used.

Binaural Direct-to-Reverberant Energy Ratio and Speaker Distance Estimation

This article addresses the problem of distance estimation using binaural hearing aid microphones in reverberant rooms. Among several distance indicators, the direct-to-reverberant energy ratio (DRR) has been shown to be more effective than other features. Therefore, we present two novel approaches to estimate the DRR of binaural signals. The first method is based on the interaural magnitude-squared coherence whereas the second approach uses stochastic maximum likelihood beamforming to estimate the power of the direct and reverberant components. The proposed DRR estimation algorithms are integrated into a distance estimation technique. When based solely on DRR, the distance estimation algorithm requires calibration where naturally the critical distance is a good calibration point. We thus propose two approaches for the calibration of the distance estimation algorithm: Informed calibration using the critical distance of the reverberant room and blind calibration using the listener's own voice. Results across various acoustical environments show the benefit of the proposed algorithms for the estimation of sound source distances up to 3 m with an estimation error of about 35 cm using informed calibration and about 1 m using the fully blind calibration strategy.

DOA Estimation in heteroscedastic noise

The paper considers direction of arrival (DOA) estimation from long-term observations in a very noisy environment. The concern is to derive methods obtaining reasonable DOAs at very low SNR. The noise is assumed zero-mean Gaussian and its variance varies in time and space, causing stationary data models to fit poorly over long observation times. Therefore a heteroscedastic Gaussian noise model is introduced where the variance varies across observations and sensors. The source amplitudes are assumed independent zero-mean complex Gaussian distributed with unknown variances (i.e. the source powers), inspiring stochastic maximum likelihood (ML) DOA estimation. The DOAs of plane waves are estimated from multi-snapshot sensor array data using sparse Bayesian learning (SBL) where the noise is estimated across both sensors and snapshots. This SBL approach is more flexible and performs better than other high-resolution methods since they cannot estimate the heteroscedastic noise process. An alternative to SBL is simple data normalization, whereby only the phase across the array is utilized. Simulations in noisy environments demonstrate that taking the heteroscedastic noise into account causes the DOA estimation to fail at lower SNR, often at 20 dB lower SNR.

Sparse Bayesian learning with multiple dictionaries

Sparse Bayesian learning (SBL) has emerged as a fast and competitive method to perform sparse processing. The SBL algorithm, which is developed using a Bayesian framework, approximately solves a non-convex optimization problem using fixed point updates. It provides comparable performance and is significantly faster than convex optimization techniques used in sparse processing. We propose a signal model which accounts for dictionary mismatch and the presence of errors in the weight vector at low signal-to-noise ratios. A fixed point update equation is derived which incorporates the statistics of mismatch and weight errors. We also process observations from multiple dictionaries. Noise variances are estimated using stochastic maximum likelihood. The derived update equations are studied quantitatively using beamforming simulations applied to direction-of-arrival (DoA). Performance of SBL using single- and multi-frequency observations, and in the presence of aliasing, is evaluated. SwellEx-96 experimental data demonstrates qualitatively the advantages of SBL.

Improving the Estimation of Sinusoidal Frequencies and Direction-of-Arrival Using Line Spectral Frequencies

In this letter we address the classic problem of estimating the frequencies of multiple sinusoids in the presence of noise. We use line spectral frequencies (LSFs) and introduce the idea of “closeness of LSFs” and propose two methods based on it. The minimum phase annihilating filter $\boldsymbol {A(z)}$ is estimated using MUSIC. The corresponding LSFs are then processed to estimate the frequencies, as opposed to estimating them from the roots that are closest to the unit circle. The closeness of LSFs criterion leads to a lower threshold when compared to MUSIC, with improvements ranging from $\text{0}$ to $\text{2}$ dB for the well-known two-sinusoids example; with pseudo-noise resampling the improvement is up to $\text{4}$ dB. When applied to direction of arrival estimation, for the example considered by Shaghaghi and Vorobyov, the threshold improves by $\text{6}$ dB and matches that given by the stochastic ML method.

DOA estimation in heteroscedastic noise

The paper considers direction of arrival (DOA) estimation from long-term observations in a noisy environment. In such an environment the noise source might evolve, causing the stationary models to fail. Therefore a heteroscedastic Gaussian noise model is introduced where the variance can vary across observations and sensors. The source amplitudes are assumed independent zero-mean complex Gaussian distributed with unknown variances (i.e. the source powers), inspiring stochastic maximum likelihood DOA estimation. The DOAs of plane waves are estimated from multi-snapshot sensor array data using sparse Bayesian learning (SBL) where the noise is estimated across both sensors and snapshots. This SBL approach is more flexible and performs better than high-resolution methods since they cannot estimate the heteroscedastic noise process. An alternative to SBL is simple data normalization, whereby only the phase across the array is utilized. Simulations demonstrate that taking the heteroscedastic noise into account improves DOA estimation.

On the stochastic restricted Liu-type maximum likelihood estimator in logistic regression model

In order to overcome multicollinearity, we propose a stochastic restricted Liu-type maximum likelihood estimator by incorporating Liu-type maximum likelihood estimator to the logistic regression model when the linear restrictions are stochastic. We also discuss the properties of the new estimator. Moreover, we give a method to choose the biasing parameter in the new estimator. Finally, a simulation study is given to show the performance of the new estimator.

A two-stage noise source identification technique based on a farfield random parametric array.

A farfield random array is implemented for noise source identification. Microphone positions are optimized, with the aid of the simulated annealing method. A two-stage localization and separation algorithm is devised on the basis of the equivalent source method (ESM). In the localization stage, the active source regions are located by using the delay-and-sum method, followed by a parametric localization procedure, stochastic maximum likelihood algorithm. Multidimensional nonlinear optimization is exploited in the bearing estimation process. In the separation stage, source amplitudes are extracted by formulating an inverse problem based on the preceding source bearings identified. The number of equivalent sources is selected to be less than that of microphones to render an overdetermined problem which can be readily solved by using the Tikhonov regularization. Alternatively, the separation problem can be augmented into an underdetermined problem which can be solved by using the compressive sensing technique. Traditionally, farfield arrays only give a relative distribution of source field. However, by using the proposed method, the acoustic variables including sound pressure, particle velocity, sound intensity, and sound power can be calculated based on ESM. Numerical and experimental results of several objective and subjective tests are presented.

Efficient direction of arrival estimation based on sparse covariance fitting criterion with modeling mismatch

This paper studies direction of arrival (DoA) estimation with an antenna array using sparse signal reconstruction (SSR). Among the existing SSR methods, the sparse covariance fitting based algorithms, which can estimate source power and noise variance naturally, are most promising. Nevertheless, they are either on-grid model based methods whose performance are sensitive to off-grid DoAs or gridless methods which are computationally demanding. In this paper, we propose an off-grid DoA estimation algorithm based on the sparse covariance fitting criterion. We first consider a scenario in which the number of snapshots is larger than the array size. An algorithm is proposed by applying an off-grid model, which takes into account the deviations between the discretized sampling grid and the true DoAs, to the sparse covariance fitting criterion. It estimates the on-grid parameters and the deviations of off-grid DoAs separately and thus is computationally efficient to implement. Then in the case where the number of snapshots is smaller than the array size, we propose to execute the DoA estimation algorithm iteratively under the stochastic maximum likelihood (SML) criterion. The estimation accuracy and computational efficiency of the proposed algorithms are demonstrated by computer simulations.

A Novel Modification of PSO Algorithm for SML Estimation of DOA.

This paper addresses the issue of reducing the computational complexity of Stochastic Maximum Likelihood (SML) estimation of Direction-of-Arrival (DOA). The SML algorithm is well-known for its high accuracy of DOA estimation in sensor array signal processing. However, its computational complexity is very high because the estimation of SML criteria is a multi-dimensional non-linear optimization problem. As a result, it is hard to apply the SML algorithm to real systems. The Particle Swarm Optimization (PSO) algorithm is considered as a rather efficient method for multi-dimensional non-linear optimization problems in DOA estimation. However, the conventional PSO algorithm suffers two defects, namely, too many particles and too many iteration times. Therefore, the computational complexity of SML estimation using conventional PSO algorithm is still a little high. To overcome these two defects and to reduce computational complexity further, this paper proposes a novel modification of the conventional PSO algorithm for SML estimation and we call it Joint-PSO algorithm. The core idea of the modification lies in that it uses the solution of Estimation of Signal Parameters via Rotational Invariance Techniques (ESPRIT) and stochastic Cramer-Rao bound (CRB) to determine a novel initialization space. Since this initialization space is already close to the solution of SML, fewer particles and fewer iteration times are needed. As a result, the computational complexity can be greatly reduced. In simulation, we compare the proposed algorithm with the conventional PSO algorithm, the classic Altering Minimization (AM) algorithm and Genetic algorithm (GA). Simulation results show that our proposed algorithm is one of the most efficient solving algorithms and it shows great potential for the application of SML in real systems.

Efficient AM Algorithms for Stochastic ML Estimation of DOA

The estimation of direction-of-arrival (DOA) of signals is a basic and important problem in sensor array signal processing. To solve this problem, many algorithms have been proposed, among which the Stochastic Maximum Likelihood (SML) is one of the most concerned algorithms because of its high accuracy of DOA. However, the estimation of SML generally involves the multidimensional nonlinear optimization problem. As a result, its computational complexity is rather high. This paper addresses the issue of reducing computational complexity of SML estimation of DOA based on the Alternating Minimization (AM) algorithm. We have the following two contributions. First using transformation of matrix and properties of spatial projection, we propose an efficient AM (EAM) algorithm by dividing the SML criterion into two components. One depends on a single variable parameter while the other does not. Second when the array is a uniform linear array, we get the irreducible form of the EAM criterion (IAM) using polynomial forms. Simulation results show that both EAM and IAM can reduce the computational complexity of SML estimation greatly, while IAM is the best. Another advantage of IAM is that this algorithm can avoid the numerical instability problem which may happen in AM and EAM algorithms when more than one parameter converges to an identical value.