Standard Maximum Likelihood Approach Research Articles

Fourier Quantum Process Tomography

The characterization of a quantum device is a crucial step in the development of quantum experiments. This is accomplished via Quantum Process Tomography, which combines the outcomes of different projective measurements to deliver a possible reconstruction of the underlying process. The tomography is typically performed by processing an overcomplete set of measurements and extracting the process matrix from maximum-likelihood estimation. Here, we introduce Fourier Quantum Process Tomography, a technique which requires a reduced number of measurements, and benchmark its performance against the standard maximum-likelihood approach. Fourier Quantum Process Tomography is based on measuring probability distributions in two conjugate spaces for different state preparations and projections. Exploiting the concept of phase retrieval, our scheme achieves a complete and robust characterization of the setup by processing a near-minimal set of measurements. We experimentally test the technique on different space-dependent polarization transformations, reporting average fidelities higher than 90% and significant computational advantage.

Multiple imputation of masked competing risks data using machine learning algorithms

The analysis of masked cause of failure data is an important area in the reliability analysis. Prior researches mostly included masking probability as a part of likelihood function to handle masked competing risks analysis which were much time-consuming, high costly and complicated computationally. To optimize time and cost and also overcome complexity of calculation, in this paper, a new two-step approach is presented which is based on multiple imputation of masked causes of failure via some machine learning algorithms. Then, in the second step, the filled-in competing risks data are analysed using standard maximum likelihood approach. For competing risks data, the superiority of the proposed method comparing with the prior ones is evaluated in ML Estimations (MLE) of Life-time parameters via several simulation studies and applying on real data. Also, sensitivity analysis for biasness versus different sample sizes is exemplified.

Linear-nonlinear cascades capture synaptic dynamics.

Short-term synaptic dynamics differ markedly across connections and strongly regulate how action potentials communicate information. To model the range of synaptic dynamics observed in experiments, we have developed a flexible mathematical framework based on a linear-nonlinear operation. This model can capture various experimentally observed features of synaptic dynamics and different types of heteroskedasticity. Despite its conceptual simplicity, we show that it is more adaptable than previous models. Combined with a standard maximum likelihood approach, synaptic dynamics can be accurately and efficiently characterized using naturalistic stimulation patterns. These results make explicit that synaptic processing bears algorithmic similarities with information processing in convolutional neural networks.

Robust modeling of additive and nonadditive variation with intuitive inclusion of expert knowledge.

We propose a novel Bayesian approach that robustifies genomic modeling by leveraging expert knowledge (EK) through prior distributions. The central component is the hierarchical decomposition of phenotypic variation into additive and nonadditive genetic variation, which leads to an intuitive model parameterization that can be visualized as a tree. The edges of the tree represent ratios of variances, for example broad-sense heritability, which are quantities for which EK is natural to exist. Penalized complexity priors are defined for all edges of the tree in a bottom-up procedure that respects the model structure and incorporates EK through all levels. We investigate models with different sources of variation and compare the performance of different priors implementing varying amounts of EK in the context of plant breeding. A simulation study shows that the proposed priors implementing EK improve the robustness of genomic modeling and the selection of the genetically best individuals in a breeding program. We observe this improvement in both variety selection on genetic values and parent selection on additive values; the variety selection benefited the most. In a real case study, EK increases phenotype prediction accuracy for cases in which the standard maximum likelihood approach did not find optimal estimates for the variance components. Finally, we discuss the importance of EK priors for genomic modeling and breeding, and point to future research areas of easy-to-use and parsimonious priors in genomic modeling.

High-dimensional Holt-Winters trend model: Fast estimation and prediction

Prediction and trend estimation are vital tasks for managers in business and industry. When the number of series increases and multiple trends arise, the estimation of trend co-movement gains importance. Indeed, it makes it possible to extract joint dynamics, measure correlations between shocks, and improve prediction performance. Unfortunately, when the model dimension increases, estimating multiple trends becomes extremely challenging, since the number of parameters tends to explode. This paper provides a closed-form result that helps simplify the prediction and estimation of the multivariate smooth-trend model, one of the state-space representations of Holt-Winters celebrated recursions. The result has several practical consequences. Indeed, we suggest a simple method that quickly maximizes the likelihood even for high-dimensional models. A Monte Carlo simulation shows the computational advantages provided by these results compared with the standard maximum likelihood approach. In addition, an empirical analysis focusing on forecasting retail sales data in the USA shows the potential of the proposed technique.

Sparse inference of the drift of a high-dimensional Ornstein–Uhlenbeck process

Given the observation of a high-dimensional Ornstein–Uhlenbeck (OU) process in continuous time, we are interested in inference on the drift parameter under a row-sparsity assumption. Towards that aim, we consider the negative log-likelihood of the process, penalized by an ℓ1-penalization (Lasso and Adaptive Lasso). We provide both finite- and large-sample results for this procedure, by means of a sharp oracle inequality, and a limit theorem in the long-time asymptotics, including asymptotic consistency for variable selection. As a by-product, we point out the fact that for the Ornstein–Uhlenbeck process, one does not need an assumption of restricted eigenvalue type in order to derive fast rates for the Lasso, while it is well-known to be mandatory for linear regression for instance. Numerical results illustrate the benefits of this penalized procedure compared to standard maximum likelihood approaches both on simulations and real-world financial data.

Hierarchical Bayesian Bradley–Terry for applications in Major League Baseball

A common problem faced in statistical inference is drawing conclusions from paired comparisons, in which two objects compete and one is declared the victor. A probabilistic approach to such a problem is the Bradley-Terry model, first studied by Zermelo in 1929 and rediscovered by Bradley and Terry in 1952. One obvious area of application for such a model is sporting events, and in particular Major League Baseball. With this in mind, we describe a hierarchical Bayesian version of Bradley-Terry suitable for use in ranking and prediction problems, and compare results from these application domains to standard maximum likelihood approaches. Our Bayesian methods outperform the MLE-based analogues, while being simple to construct, implement, and interpret.

Bayesian Versus Maximum Likelihood Estimation of Treatment Effects in Bivariate Probit Instrumental Variable Models

Bivariate probit models are a common choice for scholars wishing to estimate causal effects in instrumental variable models where both the treatment and outcome are binary. However, standard maximum likelihood approaches for estimating bivariate probit models are problematic. Numerical routines in popular software suites frequently generate inaccurate parameter estimates and even estimated correctly, maximum likelihood routines provide no straightforward way to produce estimates of uncertainty for causal quantities of interest. In this note, we show that adopting a Bayesian approach provides more accurate estimates of key parameters and facilitates the direct calculation of causal quantities along with their attendant measures of uncertainty.

Using copulas to estimate the coefficient of a binary endogenous regressor in a Poisson regression: Application to the effect of insurance on doctor visits.

This paper presents a copula-based method for identifying and estimating the coefficient of a binary endogenous regressor in a Poisson regression. The method offers advantages over existing approaches. Most importantly, it relies upon standard maximum likelihood approaches, and it does not require numerical integration. Further, as part of its implementation, the method provides a convenient test for the presence of endogeneity. The empirical application investigates the effect of insurance status (a binary measure) on doctor visits (a count measure).

Land Cover and Soil Type Mapping From Spaceborne PolSAR Data at L-Band With Probabilistic Neural Network

This paper evaluates performance of fully polarimetric SAR (PolSAR) data in several land cover mapping studies in the boreal forest environment, taking advantage of the high canopy penetration capability at L-band. The studies included multiclass land cover mapping, forest-nonforest delineation, and classification of soil type under vegetation. PolSAR data used in the study were collected by the ALOS PALSAR sensor in 2006-2007 over a managed boreal forest site in Finland. A supervised classification approach using selected polarimetric features in the framework of probabilistic neural network (PNN) was adopted in the study. It has no assumptions about statistics of the polarimetric features, using nonparametric estimation of probability distribution functions instead. The PNN-based method improved classification accuracy compared with standard maximum-likelihood approach. The improvement was considerably strong for soil type mapping under vegetation, indicating notable non-Gaussian effects in the PolSAR data even at L-band. The classification performance was strongly dependent on seasonal conditions. The PolSAR feature data set was further modified to include a number of recently proposed polarimetric parameters (surface scattering fraction and scattering diversity), reducing the computational complexity at practically no loss in the classification accuracy. The best obtained accuracies of up to 82.6% in five-class land cover mapping and more than 90% in forest-nonforest mapping in wall-to-wall validation indicate suitability of PolSAR data for wide-area land cover and forest mapping.

Automated measurement of redshifts from mid-infrared low-resolution spectroscopy

We present a new SED-fitting based routine for redshift determination that is optimised for mid-infrared (MIR) low-resolution spectroscopy. Its flexible template scaling increases the sensitivity to slope changes and small scale features in the spectrum, while a new selection algorithm called Maximum Combined Pseudo-Likelihood (MCPL) provides increased accuracy and a lower number of outliers compared to the standard maximum-likelihood (ML) approach. Unlike ML, MCPL searches for local (instead of absolute) maxima of a 'pseudo-likelihood' (PL) function, and combines results obtained for all the templates in the library to weed out spurious redshift solutions. The capabilities of MCPL are demonstrated by comparing its results to those of regular ML and to the optical spectroscopic redshifts of a sample of 491 Spitzer/IRS spectra from sources at 0<z<3.7. MCPL achieves a redshift accuracy dz/(1+z)<0.005 for 78% of the galaxies in the sample compared to 68% for ML. The rate of outliers (dz/(1+z)>0.02) is 14% for MCPL and 22% for ML. chi^2 values for ML solutions are found to correlate with the SNR of the spectra, but not with redshift accuracy. By contrast, the peak value of the normalised combined PL (gamma) is found to provide a good indication on the reliability of the MCPL solution for individual sources. The accuracy and reliability of the redshifts depends strongly on the MIR SED. Sources with significant polycyclic aromatic hydrocarbon emission obtain much better results compared to sources dominated by AGN continuum. Nevertheless, for a given gamma the frequency of accurate solutions and outliers is largely independent on their SED type. This reliability indicator for MCPL solutions allows to select subsamples with highly reliable redshifts. In particular, a gamma>0.15 threshold retains 79% of the sources with dz/(1+z)<0.005 while reducing the outlier rate to 3.8% (abridged).

Estimation of Ordered Response Models with Sample Selection

We introduce two new Stata commands for the estimation of an ordered response model with sample selection. The opsel command uses a standard maximum-likelihood approach to fit a parametric specification of the model where errors are assumed to follow a bivariate Gaussian distribution. The snpopsel command uses the semi-nonparametric approach of Gallant and Nychka (1987, Econometrica 55: 363–390) to fit a semiparametric specification of the model where the bivariate density function of the errors is approximated by a Hermite polynomial expansion. The snpopsel command extends the set of Stata routines for semi-nonparametric estimation of discrete response models. Compared to the other semi-nonparametric estimators, our routine is relatively faster because it is programmed in Mata. In addition, we provide new postestimation routines to compute linear predictions, predicted probabilities, and marginal effects. These improvements are also extended to the set of semi-nonparametric Stata commands originally written by Stewart (2004, Stata Journal 4: 27–39) and De Luca (2008, Stata Journal 8: 190–220). An illustration of the new opsel and snpopsel commands is provided through an empirical application on self-reported health with selectivity due to sample attrition.

Estimation of Ordered Response Models with Sample Selection

We introduce two new Stata commands for the estimation of an ordered response model with sample selection. The opsel command uses a standard maximum likelihood (ML) approach to fit a parametric specification of the model where errors are assumed to follow a bivariate Gaussian distribution. The snpopsel command uses the semi-nonparametric (SNP) approach of Gallant and Nychka (1987) to fit a semiparametric specification of the model where the bivariate density function of the errors is approximated by a Hermite polynomial expansion. The snpopsel command extends the set of Stata routines for SNP estimation of discrete response models. Compared to the other SNP estimators, our routine is relatively faster because it is programmed in MATA. In addition, we provide new post-estimation routines to compute linear predictions, predicted probabilities and marginal effects. These improvements are also extended to the set of SNP Stata commands originally written by Stewart (2004) and De Luca (2008). An illustration of the new opsel and snpopsel commands is provided through an empirical application on self-reported health with selectivity due to sample attrition.

Biomarker discovery using targeted maximum-likelihood estimation: application to the treatment of antiretroviral-resistant HIV infection.

Researchers in clinical science and bioinformatics frequently aim to learn which of a set of candidate biomarkers is important in determining a given outcome, and to rank the contributions of the candidates accordingly. This article introduces a new approach to research questions of this type, based on targeted maximum-likelihood estimation of variable importance measures.The methodology is illustrated using an example drawn from the treatment of HIV infection. Specifically, given a list of candidate mutations in the protease enzyme of HIV, we aim to discover mutations that reduce clinical virologic response to antiretroviral regimens containing the protease inhibitor lopinavir. In the context of this data example, the article reviews the motivation for covariate adjustment in the biomarker discovery process. A standard maximum-likelihood approach to this adjustment is compared with the targeted approach introduced here. Implementation of targeted maximum-likelihood estimation in the context of biomarker discovery is discussed, and the advantages of this approach are highlighted. Results of applying targeted maximum-likelihood estimation to identify lopinavir resistance mutations are presented and compared with results based on unadjusted mutation-outcome associations as well as results of a standard maximum-likelihood approach to adjustment.The subset of mutations identified by targeted maximum likelihood as significant contributors to lopinavir resistance is found to be in better agreement with the current understanding of HIV antiretroviral resistance than the corresponding subsets identified by the other two approaches. This finding suggests that targeted estimation of variable importance represents a promising approach to biomarker discovery.

Representation of photon limited data in emission tomography using origin ensembles

Representation and reconstruction of data obtained by emission tomography scanners are challenging due to high noise levels in the data. Typically, images obtained using tomographic measurements are represented using grids. In this work, we define images as sets of origins of events detected during tomographic measurements; we call these origin ensembles (OEs). A state in the ensemble is characterized by a vector of 3N parameters Y, where the parameters are the coordinates of origins of detected events in a three-dimensional space and N is the number of detected events. The 3N-dimensional probability density function (PDF) for that ensemble is derived, and we present an algorithm for OE image estimation from tomographic measurements. A displayable image (e.g. grid based image) is derived from the OE formulation by calculating ensemble expectations based on the PDF using the Markov chain Monte Carlo method. The approach was applied to computer-simulated 3D list-mode positron emission tomography data. The reconstruction errors for a 10 000 000 event acquisition for simulated ranged from 0.1 to 34.8%, depending on object size and sampling density. The method was also applied to experimental data and the results of the OE method were consistent with those obtained by a standard maximum-likelihood approach. The method is a new approach to representation and reconstruction of data obtained by photon-limited emission tomography measurements.

Bayesian Adaptive Inference and Adaptive Training

<para xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> Large-vocabulary speech recognition systems are often built using found data, such as broadcast news. In contrast to carefully collected data, found data normally contains multiple acoustic conditions, such as speaker or environmental noise. Adaptive training is a powerful approach to build systems on such data. Here, transforms are used to represent the different acoustic conditions, and then a canonical model is trained given this set of transforms. This paper describes a Bayesian framework for adaptive training and inference. This framework addresses some limitations of standard maximum-likelihood approaches. In contrast to the standard approach, the adaptively trained system can be directly used in unsupervised inference, rather than having to rely on initial hypotheses being present. In addition, for limited adaptation data, robust recognition performance can be obtained. The limited data problem often occurs in testing as there is no control over the amount of the adaptation data available. In contrast, for adaptive training, it is possible to control the system complexity to reflect the available data. Thus, the standard point estimates may be used. As the integral associated with Bayesian adaptive inference is intractable, various marginalization approximations are described, including a variational Bayes approximation. Both batch and incremental modes of adaptive inference are discussed. These approaches are applied to adaptive training of maximum-likelihood linear regression and evaluated on a large-vocabulary speech recognition task. Bayesian adaptive inference is shown to significantly outperform standard approaches. </para>

Bayesian estimation applied to multiple species

Observed data are often contaminated by undiscovered interlopers, leading to biased parameter estimation. Here we present BEAMS (Bayesian estimation applied to multiple species) which significantly improves on the standard maximum likelihood approach in the case where the probability for each data point being “pure” is known. We discuss the application of BEAMS to future type-Ia supernovae (SNIa) surveys, such as LSST, which are projected to deliver over a million supernovae light curves without spectra. The multiband light curves for each candidate will provide a probability of being Ia (pure) but the full sample will be significantly contaminated with other types of supernovae and transients. Given a sample of N supernovae with mean probability, ⟨P⟩, of being Ia, BEAMS delivers parameter constraints equal to N⟨P⟩ spectroscopically confirmed SNIa. In addition BEAMS can be simultaneously used to tease apart different families of data and to recover properties of the underlying distributions of those families (e.g. the type-Ibc and II distributions). Hence BEAMS provides a unified classification and parameter estimation methodology which may be useful in a diverse range of problems such as photometric redshift estimation or, indeed, any parameter estimation problem where contamination is an issue.

SAMPLING VARIABILITY AND ESTIMATES OF DENSITY DEPENDENCE: A COMPOSITE-LIKELIHOOD APPROACH

It is well known that sampling variability, if not properly taken into account, affects various ecologically important analyses. Statistical inference for stochastic population dynamics models is difficult when, in addition to the process error, there is also sampling error. The standard maximum-likelihood approach suffers from large computational burden. In this paper, I discuss an application of the composite-likelihood method for estimation of the parameters of the Gompertz model in the presence of sampling variability. The main advantage of the method of composite likelihood is that it reduces the computational burden substantially with little loss of statistical efficiency. Missing observations are a common problem with many ecological time series. The method of composite likelihood can accommodate missing observations in a straightforward fashion. Environmental conditions also affect the parameters of stochastic population dynamics models. This method is shown to handle such nonstationary population dynamics processes as well. Many ecological time series are short, and statistical inferences based on such short time series tend to be less precise. However, spatial replications of short time series provide an opportunity to increase the effective sample size. Application of likelihood-based methods for spatial time-series data for population dynamics models is computationally prohibitive. The method of composite likelihood is shown to have significantly less computational burden, making it possible to analyze large spatial time-series data. After discussing the methodology in general terms, I illustrate its use by analyzing a time series of counts of American Redstart (Setophaga ruticilla) from the Breeding Bird Survey data, San Joaquin kit fox (Vulpes macrotis mutica) population abundance data, and spatial time series of Bull trout (Salvelinus confluentus) redds count data.

Assessing excess lifetime risk for disease after radiation exposure.

In this paper, we define the excess lifetime risk for a disease due to radiation exposure, and we evaluate the excess lifetime risk for an exposed group with the same age and with a mixture of ages. We propose two methods to estimate the excess lifetime risk: the standard maximum likelihood approach and a Monte-Carlo approach. Both approaches allow us to construct confidence intervals for the excess lifetime risk and perform hypothesis testing to compare two excess risks. The maximum likelihood approach is straightforward and computationally simple; however, it may not provide good approximations when the normality assumption is violated. The Monte-Carlo approach is always reliable, but often involves heavier computations and additional programming. These two approaches are illustrated and discussed based on a pooled analysis of five cohorts exposed to external radiation and followed up for subsequent thyroid cancer incidence.

Communications with chaotic time series: probabilistic methods for noise reduction

The paper deals with noise decontamination of chaotic time series under the assumption that some a priori information about the system which produced the time series is known in advance. We show that this a priori information can be quite naturally used in standard maximum likelihood approaches. Focusing on low complexity implementations we derive different quasioptimal maximum-likelihood solutions aimed at off-line and on-line noise cleaning. The obtained results show attractive capabilities for on-line and low-cost implementation. Copyright © 1999 John Wiley & Sons, Ltd.