Constrained Maximum Likelihood Estimation Research Articles

A Choice Model with a Diverging Choice Set for POI Data Analysis

A point of interest (POI) is a geographical location, that might carry interest for the public. A POI provides a convenient way to register people's locations through mobile devices, which leads to POI data. POI data contain accurate location information and are extremely valuable for location based services (LBS). Accordingly, principled statistical methods, which can be used for regression and/or prediction are required. To partially fulfill this theoretical gap, we propose a conditional logit approach for POI choice analysis. This new model is a natural extension of the classical choice model (McFadden, 1974, 1978) but with two key characteristics. First, POIs located far away from the current position are less likely to be selected as the next POI choice. As a result, the distance (or its appropriate transformation) between the current position and the next POI candidate is an important predictor and should be included in the model. Second, the classical choice model considers a finite choice set. By contrast, the new model studies a diverging choice set, mainly because the total number of POI locations in practice is typically large. The diverging choice set produces an expensive computation of the maximum likelihood estimation (MLE). To alleviate computational costs, we further propose a constrained maximum likelihood estimation (CMLE) method. Compared with MLE, CMLE utilizes only those POIs located within a reasonable distance. This prioritization leads to a significant reduction in computation at a reasonable efficiency loss. To demonstrate the finite sample performance of the method, numerical studies based on both simulated and real datasets are presented.

Kernel-based nonparametric estimation methods for a periodic replacement problem with minimal repair

In this article, we consider nonparametric estimation methods for a periodic replacement problem with minimal repair, where the expected cumulative number of failures (minimal repairs) is unknown. To construct the confidence interval of an estimator of the optimal periodic replacement time which minimizes the long-run average cost per unit time, we apply two kernel-based bootstrap estimation methods and three replication techniques for bootstrap samples, to estimate the optimal periodic replacement time under incomplete knowledge on the failure time distribution. In simulation experiments, we compare those results with the well-known constrained nonparametric maximum likelihood estimate and some parametric models. We also conduct the field data analysis based on an actual minimal repair data and refer to an applicability of our methods.

Constrained Maximum Likelihood Estimation of Relative Abundances of Protein Conformation in a Heterogeneous Mixture from Small Angle X-Ray Scattering Intensity Measurements.

In this paper, we describe a model for maximum likelihood estimation (MLE) of the relative abundances of different conformations of a protein in a heterogeneous mixture from small angle X-ray scattering (SAXS) intensities. To consider cases where the solution includes intermediate or unknown conformations, we develop a subset selection method based on k-means clustering and the Cramér-Rao bound on the mixture coefficient estimation error to find a sparse basis set that represents the space spanned by the measured SAXS intensities of the known conformations of a protein. Then, using the selected basis set and the assumptions on the model for the intensity measurements, we show that the MLE model can be expressed as a constrained convex optimization problem. Employing the adenylate kinase (ADK) protein and its known conformations as an example, and using Monte Carlo simulations, we demonstrate the performance of the proposed estimation scheme. Here, although we use 45 crystallographically determined experimental structures and we could generate many more using, for instance, molecular dynamics calculations, the clustering technique indicates that the data cannot support the determination of relative abundances for more than 5 conformations. The estimation of this maximum number of conformations is intrinsic to the methodology we have used here.

Fusion of hard and soft information in nonparametric density estimation

This paper discusses univariate density estimation in situations when the sample (hard information) is supplemented by “soft” information about the random phenomenon. These situations arise broadly in operations research and management science where practical and computational reasons severely limit the sample size, but problem structure and past experiences could be brought in. In particular, density estimation is needed for generation of input densities to simulation and stochastic optimization models, in analysis of simulation output, and when instantiating probability models. We adopt a constrained maximum likelihood estimator that incorporates any, possibly random, soft information through an arbitrary collection of constraints. We illustrate the breadth of possibilities by discussing soft information about shape, support, continuity, smoothness, slope, location of modes, symmetry, density values, neighborhood of known density, moments, and distribution functions. The maximization takes place over spaces of extended real-valued semicontinuous functions and therefore allows us to consider essentially any conceivable density as well as convenient exponential transformations. The infinite dimensionality of the optimization problem is overcome by approximating splines tailored to these spaces. To facilitate the treatment of small samples, the construction of these splines is decoupled from the sample. We discuss existence and uniqueness of the estimator, examine consistency under increasing hard and soft information, and give rates of convergence. Numerical examples illustrate the value of soft information, the ability to generate a family of diverse densities, and the effect of misspecification of soft information.

Confidence interval construction for the difference between two correlated proportions with missing observations

Under the assumption of missing at random, eight confidence intervals (CIs) for the difference between two correlated proportions in the presence of incomplete paired binary data are constructed on the basis of the likelihood ratio statistic, the score statistic, the Wald-type statistic, the hybrid method incorporated with the Wilson score and Agresti–Coull (AC) intervals, and the Bootstrap-resampling method. Extensive simulation studies are conducted to evaluate the performance of the presented CIs in terms of coverage probability and expected interval width. Our empirical results evidence that the Wilson-score-based hybrid CI and the Wald-type CI together with the constrained maximum likelihood estimates perform well for small-to-moderate sample sizes in the sense that (i) their empirical coverage probabilities are quite close to the prespecified confidence level, (ii) their expected interval widths are shorter, and (iii) their ratios of the mesial non-coverage to non-coverage probabilities lie in interval [0.4, 0.6]. An example from a neurological study is used to illustrate the proposed methodologies.

Constrained Maximum Likelihood Estimation of the Abundances of Protein Conformation in a Heterogeneous Structural Ensemble from Small Angle X-ray Scattering Intensity Measurements

A method is described for maximum likelihood estimation (MLE) of the relative abundances of different conformations of a protein in a heterogeneous mixture based on small angle X-ray scattering (SAXS) intensities.This approach is of particular interest in situations where there are unknown, intermediate conformations, for instance, during catalytic cycling of a protein. First, an ensemble of structures is generated using molecular dynamics, crystallography or other technique. This ensemble is then clustered into sub-sets based on k-means clustering and the Cramer-Rao bound on the mixture coefficient estimation error. A sparse basis set that represents the space spanned by the measured SAXS intensities of the conformations of a protein is then generated from representative members of each cluster. Based on a statistical model for the intensity measurements, we show that the MLE approach can be expressed as a constrained convex optimization problem. Starting with a basis set generated from known conformations of the enzyme, adenylate kinase (ADK), we carried out Monte Carlo simulations to assess the performance of the proposed estimation scheme. We demonstrate the utility of the approach by identification of dominant conformations under different solution conditions and provide estimates of the abundance of minor species as a function of concentration of different ligands.

An MPEC estimator for misclassification models

In this paper, we propose a constrained maximum likelihood estimator for misclassification models, by formulating the estimation as an MPEC (Mathematical Programming with Equilibrium Constraints) problem. Our approach improves the numerical accuracy and avoids the singularity problem. Monte Carlo simulations confirm that the proposed estimator reduces bias and standard deviation of the estimator, especially when the sample is small/medium and/or the dimension of latent variable is large.

Testing uniformity based on new entropy estimators

In this paper, we first introduce new entropy estimators for distributions with known and bounded supports. Our estimators are obtained by using constrained maximum likelihood estimation of cumulative distribution function for absolutely continuous distributions with known and bounded supports. We prove the consistency of our estimators. Then, we propose uniformity tests based on the proposed entropy estimators and compare their powers with the powers of other tests of uniformity. Our simulation results show that the proposed entropy estimators perform well in estimating entropy and testing uniformity.

Higher order inference for stress–strength reliability with independent Burr-type X distributions

In this paper, a small-sample asymptotic method is proposed for higher order inference in the stress–strength reliability model, R=P(Y<X), where X and Y are distributed independently as Burr-type X distributions. In a departure from the current literature, we allow the scale parameters of the two distributions to differ, and the likelihood-based third-order inference procedure is applied to obtain inference for R. The difficulty of the implementation of the method is in obtaining the the constrained maximum likelihood estimates (MLE). A penalized likelihood method is proposed to handle the numerical complications of maximizing the constrained likelihood model. The proposed procedures are illustrated using a sample of carbon fibre strength data. Our results from simulation studies comparing the coverage probabilities of the proposed small-sample asymptotic method with some existing large-sample asymptotic methods show that the proposed method is very accurate even when the sample sizes are small.

Two new methods for non-inferiority testing of the ratio in matched-pair setting

Non-inferiority of one treatment to another based on ratio is a common issue in medical research. Nam and Blackwelder derived a Fieller-type statistic from constrained maximum likelihood estimation of nuisance parameters. Although it improved that of Lachenbruch and Lynch, its size still exceeds the claimed nominal level alpha for some cases. This is in accordance with Sidik who has found that the asymptotic tests are inaccurate in testing a hypothesis of a non-zero difference between marginal probabilities. In this paper, we extend Sidik’s exact tests to non-inferiority in ratio, and propose two new methods—one based on the standard \(p\) value and the other from restricted Bayesian estimation using an approximate \(p\) value. Our tests are both based on only one point of the two-dimension nuisance parameter space for accuracy improvement and computational purposes. The sizes and powers of our tests are considered. Simulation results suggest that our tests can control the type I error rates well with competitive powers while the Fieller-type statistic cannot for some cases be investigated.

Convergence and Stability of a Class of Iteratively Re-weighted Least Squares Algorithms for Sparse Signal Recovery in the Presence of Noise.

In this paper, we study the theoretical properties of a class of iteratively re-weighted least squares (IRLS) algorithms for sparse signal recovery in the presence of noise. We demonstrate a one-to-one correspondence between this class of algorithms and a class of Expectation-Maximization (EM) algorithms for constrained maximum likelihood estimation under a Gaussian scale mixture (GSM) distribution. The IRLS algorithms we consider are parametrized by 0 < ν ≤ 1 and ε > 0. The EM formalism, as well as the connection to GSMs, allow us to establish that the IRLS(ν, ε) algorithms minimize ε-smooth versions of the ℓ ν 'norms'. We leverage EM theory to show that, for each 0 < ν ≤ 1, the limit points of the sequence of IRLS(ν, ε) iterates are stationary point of the ε-smooth ℓ ν 'norm' minimization problem on the constraint set. Finally, we employ techniques from Compressive sampling (CS) theory to show that the class of IRLS(ν, ε) algorithms is stable for each 0 < ν ≤ 1, if the limit point of the iterates coincides the global minimizer. For the case ν = 1, we show that the algorithm converges exponentially fast to a neighborhood of the stationary point, and outline its generalization to super-exponential convergence for ν < 1. We demonstrate our claims via simulation experiments. The simplicity of IRLS, along with the theoretical guarantees provided in this contribution, make a compelling case for its adoption as a standard tool for sparse signal recovery.

A MPEC Estimator for Misclassification Models

In this paper, we propose a constrained maximum likelihood estimator for misclassification models, by formulating the estimation as an MPEC (Mathematical Programming with Equilibrium Constraints) problem. Our approach improves the numerical accuracy and avoids the singularity problem. Monte Carlo simulations confirm that the proposed estimator reduces bias and standard deviation of the estimator, especially when the sample is small/medium and/or the dimension of matrix is large.

Constrained inference for generalized linear models with incomplete covariate data

Missing data are common in many experiments, including surveys, clinical trials, epidemiological studies, and environmental studies. Unconstrained likelihood inferences for generalized linear models (GLMs) with nonignorable missing covariates have been studied extensively in the literature. However, parameter orderings or constraints may occur naturally in practice, and thus the efficiency of a statistical method may be improved by incorporating parameter constraints into the likelihood function. In this paper, we consider constrained inference for analysing GLMs with nonignorable missing covariates under linear inequality constraints on the model parameters. Specifically, constrained maximum likelihood (ML) estimation is based on the gradient projection expectation maximization approach. Further, we investigate the asymptotic null distribution of the constrained likelihood ratio test (LRT). Simulations study the empirical properties of the constrained ML estimators and LRTs, which demonstrate improved precision of these constrained techniques. An application to contaminant levels in an environmental study is also presented.

On the performance of the physicality-constrained maximum-likelihood estimation of Stokes vector

We address the estimation of the Stokes vectors taking into account the physical realizability constraint. We propose a fast method for computing the constrained maximum-likelihood (CML) estimator for any measurement matrix, and we compare its performance with the classical empirical physicality-constrained estimator. We show that when the measurement matrix is based on four polarization states spanning a regular tetrahedron on the Poincaré sphere, the two estimators are very similar, but the CML provides a better estimation of the intensity. For an arbitrary measurement matrix, the CML estimator does not always yield better estimation performance than the empirical one: their comparative performances depend on the measurement matrix, the actual Stokes vector and the signal-to-noise ratio.

Constrained Maximum Likelihood Estimation of Cell Probabilities Under Local Odds Ratio Constraints

Constrained Maximum Likelihood Estimation of Cell Probabilities Under Local Odds Ratio Constraints

Dynamic Inventory–Pricing Control Under Backorder: Demand Estimation and Policy Optimization

Inventory-based dynamic pricing has become a common operations strategy in practice and has received considerable attention from the research community. From an implementation perspective, it is desirable to design a simple policy like a base-stock list-price (BSLP) policy. The existing research on this problem often imposes restrictive conditions to ensure the optimality of a BSLP policy, which limits its applicability in practice. In this paper, we analyze the dynamic inventory and pricing control problem in which the demand follows a generalized additive model (GAM). The GAM overcomes the limitations of several demand models commonly used in the literature, but introduces analytical challenges in analyzing the dynamic program. Via a variable transformation approach, we identify a new set of technical conditions under which a BSLP policy is optimal. These conditions are easy to verify because they depend only on the location and scale parameters of demand as functions of price and are independent of the cost parameters or the distribution of the random demand component. Moreover, although a BSLP policy is optimal under these conditions, the optimal price may not be monotone decreasing in the inventory level. We further demonstrate our results by applying a constrained maximum likelihood estimation procedure to simultaneously estimate the demand function and verify the optimality of a BSLP policy on a retail data set.

Estimation in functional regression for general exponential families

This paper studies a class of exponential family models whose canonical parameters are specified as linear functionals of an unknown infinite-dimensional slope function. The optimal minimax rates of convergence for slope function estimation are established. The estimators that achieve the optimal rates are constructed by constrained maximum likelihood estimation with parameters whose dimension grows with sample size. A change-of-measure argument, inspired by Le Cam's theory of asymptotic equivalence, is used to eliminate the bias caused by the nonlinearity of exponential family models.

Statistical inference of risk ratio in a correlated $$2 \times 2$$ table with structural zero

This paper studies a compound interval hypothesis about risk ratio in an incomplete correlated $$2\times 2$$ table. Asymptotic test statistics of the Wald-type and the logarithmic transformation are proposed, with methods of the sample estimation and the constrained maximum likelihood estimation (CMLE) being considered. Score test statistic is also considered for the interval hypothesis. The approximate sample size formulae required for a specific power for these tests are presented. Simulation results suggest that the logarithmic transformation test based on CMLE method outperforms the other tests in terms of true type I error rate. A real example is used to illustrate the proposed methods.

A general binomial regression model to estimate standardized risk differences from binary response data

Estimates of absolute risks and risk differences are necessary for evaluating the clinical and population impact of biomedical research findings. We have developed a linear-expit regression model (LEXPIT) to incorporate linear and nonlinear risk effects to estimate absolute risk from studies of a binary outcome. The LEXPIT is a generalization of both the binomial linear and logistic regression models. The coefficients of the LEXPIT linear terms estimate adjusted risk differences, whereas the exponentiated nonlinear terms estimate residual odds ratios. The LEXPIT could be particularly useful for epidemiological studies of risk association, where adjustment for multiple confounding variables is common. We present a constrained maximum likelihood estimation algorithm that ensures the feasibility of risk estimates of the LEXPIT model and describe procedures for defining the feasible region of the parameter space, judging convergence, and evaluating boundary cases. Simulations demonstrate that the methodology is computationally robust and yields feasible, consistent estimators. We applied the LEXPIT model to estimate the absolute 5-year risk of cervical precancer or cancer associated with different Pap and human papillomavirus test results in 167,171 women undergoing screening at Kaiser Permanente Northern California. The LEXPIT model found an increased risk due to abnormal Pap test in human papillomavirus-negative that was not detected with logistic regression. Our R package blm provides free and easy-to-use software for fitting the LEXPIT model.

Forecasting warranty claims for recently launched products

Forecasting warranty claims for recently launched products that have short histories of claim records is vitally important for manufacturers in preparing their fiscal plans. Since the amount of historical claim data for such products is not large enough, developing forecasting models with good performance has been a difficult problem.The objective of this paper is to develop an algorithm for forecasting the number of warranty claims of recently launched products. A two-phase modelling algorithm is developed: in Phase I, we estimate the upper and the lower bounds of the warranty claim rates of the reference products that have been in the market for a longer time; in Phase II, we build forecasting models for the recently launched products and assume that their future claim rates are subject to the bound constraints derived from Phase I. Based on this algorithm, we use the NHPP (non-homogeneous Poisson process) and the constrained maximum likelihood estimation to build forecasting models on artificially generated data as well as warranty claim data collected from an electronics manufacturer. The results show that the proposed algorithm outperforms commonly used NHPP models.