Direct Maximum Likelihood Estimation Research Articles

Robust data integration from multiple external sources for generalized linear models with binary outcomes.

We aim to estimate parameters in a generalized linear model (GLM) for a binary outcome when, in addition to the raw data from the internal study, more than 1 external study provides summary information in the form of parameter estimates from fitting GLMs with varying subsets of the internal study covariates. We propose an adaptive penalization method that exploits the external summary information and gains efficiency for estimation, and that is both robust and computationally efficient. The robust property comes from exploiting the relationship between parameters of a GLM and parameters of a GLM with omitted covariates and from downweighting external summary information that is less compatible with the internal data through a penalization. The computational burden associated with searching for the optimal tuning parameter for the penalization is reduced by using adaptive weights and by using an information criterion when searching for the optimal tuning parameter. Simulation studies show that the proposed estimator is robust against various types of population distribution heterogeneity and also gains efficiency compared to direct maximum likelihood estimation. The method is applied to improve a logistic regression model that predicts high-grade prostate cancer making use of parameter estimates from 2 external models.

The Role of Micro Finance Institutions on the Development of Micro Enterprises (MEs) in Sri Lanka

Aim: The study investigates the impact of microfinance practices on the growth of micro-enterprises (MEs) concerning the Central Province, Sri Lanka. It has been discovered that the growth of microbusinesses is dependent on gender, age, education level and nature of the industry.
 Design/Methodology/Approach: Multinomial Logistic Regression models was applied in this study. Multinomial logistic regression is frequently used for the analysis of categorical response data with continuous or categorical explanatory variables. Parameter estimates are usually obtained through direct maximum likelihood estimation. Two models were used to test the hypothesis concerning the three practices, micro-credit, training and advisory services. Primary data were obtained from 200 registered microenterprises (ME) owners in the Central Province through convenience sampling methods. Data collection was conducted using a self-structured questionnaire.
 Results and Conclusion: According to the results, microfinance practices have a significant relationship with the development of MEs, while Microcredit and advisory services have a significant impact on the development of MEs. Training programmes have not significantly impacted on the development of MEs. The research concludes that microfinance as a whole educates and helps to develop micro financed micro-scale enterprise businesses and positively impact those families in the Central Province, Sri Lanka.
 Research limitations/implications: Data were limited to select only one province in Sri Lanka out of nine using a self-structured questionnaire. Also considering the response rate and sample size, there are limitations to generalize the findings. This research was restricted to three variables micro-credit; Training and Advisory services impact of other factors that can influence the growth of MEs did not fall under the scope of this study.
 Originality and Value: The impact of microfinance practices on MEs growth in Central Province in Sri Lanka is an under-researched area of study. The findings of this study can act as a guideline in the future for decision-makers to identify factors that influence more on MEs development.

A Comparison of Maximum Likelihood Estimations for Normal Mean under Right Censoring

For efficiently estimating the normal mean ([Formula: see text]) under right censoring (threshold =[Formula: see text], [Formula: see text] is known), we compare two approaches within the maximum likelihood estimation (MLE) framework. Approach I is a hierarchical MLE for which only the empirical censoring probability is utilized. Approach II is the direct MLE for which expectation-maximization (EM) algorithm is applied to all individual observations. We use discrete approximation to explain that the asymptotic variance of Approach II estimate equals the inverse Fisher information calculated from the full log-likelihood. We prove that Approach II gives a uniformly smaller asymptotic variance than Approach I and the variance ratio is a decreasing function of [Formula: see text]. We further prove some supportive results and graphically demonstrate that EM algorithm monotonically converges to the unique MLE.

Study protocol for a randomised controlled trial evaluating an evidence-based, stepped and coordinated care service model for mental disorders (RECOVER)

IntroductionHealthcare systems around the world are looking for solutions to the growing problem of mental disorders. RECOVER is the synonym for an evidence-based, stepped and cross-sectoral coordinated care service model...

Blind single image super-resolution with a mixture of deep networks

Existing deep neural network based image super-resolution (SR) methods are mostly designed for non-blind cases, where the blur kernel used to generate the low-resolution (LR) images is assumed to be known and fixed. However, this assumption does not hold in many real scenarios. Motivated by the observation that SR of LR images generated by different blur kernels are essentially different but also correlated, we propose a mixture model of deep networks, which is capable of clustering SR tasks of different blur kernels into a set of groups. Each group is composed of correlated SR tasks with similar blur kernels and can be effectively handled by a combination of specific networks in the mixture model. To achieve automatic SR tasks clustering and network selection, we model the blur kernel with a latent variable, which is inferred from the input image by an encoder network. Since the ground-truth of the latent variable is unknown in the training stage, we initialize the encoder network by pre-training it on the blur kernel classification task to avoid trivial solutions. To jointly train the mixture model and the encoder network, we further derive a lower bound of the likelihood function, which circumvents the intractability in direct maximum likelihood estimation. Extensive evaluations are performed on benchmark data sets and validate the effectiveness of the proposed method.

The t linear mixed model: model formulation, identifiability and estimation

The robustness of the t linear mixed model (tLMM) has been proved and exploited in many applications. Various publications emerged with the aim of proving superiority with respect to traditional linear mixed models, extending to more general settings and proposing more efficient estimation methods. However, little attention has been paid to the mathematical properties of the model itself and to the evaluation of the proposed estimation methods. In this paper we perform an in-depth analysis of the tLMM, evaluating a direct maximum likelihood estimation method via an intensive simulation study and investigating some identifiability properties. The theoretical findings are illustrated through an application to a dataset collected from a sleep trial.

Inference for ETAS models with non-Poissonian mainshock arrival times

The Hawkes process is a widely used statistical model for point processes which produce clustered event times. A specific version known as the ETAS model is used in seismology to forecast earthquake arrival times under the assumption that mainshocks follow a Poisson process, with aftershocks triggered via a parametric kernel function. However, this Poissonian assumption contradicts several aspects of seismological theory which suggest that the arrival time of mainshocks instead follows alternative renewal distributions such as the Gamma or Brownian Passage Time. We hence show how the standard ETAS/Hawkes process can be extended to allow for non-Poissonian distributions by introducing a dependence based on the underlying process’ behaviour. Direct maximum likelihood estimation of the resulting models is not computationally feasible in the general case, so we also present a novel Bayesian MCMC algorithm for efficient estimation using a latent variable representation.

Likelihood estimation of drug occupancy for brain PET studies

Neuroimaging with PET is unique in its capability to measure in vivo the occupancy of a drug. The occupancy is typically obtained by conducting PET measurements before and after administration of the drug. For radioligands for which no reference region exists, however, the only established procedure to estimate the occupancy from these data is via linear regression analysis, forming the basis for the so-called Lassen plot. There are several reasons why simple linear regression analysis is not ideal for analyzing these data, including regression attenuation and correlated errors.Here, we propose the use of Likelihood Estimation of Occupancy (LEO) in such a situation. Similar to the Lassen plot, LEO uses the total distribution volume estimates at baseline and at block condition as input, but estimates the non-displaceable distribution volume (VND) and fractional occupancy (Δ) via direct maximum likelihood estimation (MLE).This study outlines the rationale for using MLE to estimate Δ and VND from PET data, and evaluates its performance in relation to the Lassen Plot via two separate simulation experiments. Finally, LEO and Lassen plot are applied to a PET dataset acquired with [11C]WAY-100635.LEO can exploit the covariance structure of the data to improve the accuracy and precision of the estimates of Δ and VND. Theoretically, the covariance matrix can be extracted from a test-retest dataset for the radioligand at hand. Several procedures to estimate the covariance matrix were considered as part of the simulation experiments, and the effect of the test-retest sample size was also assessed.The results are conclusive in that MLE can be used to estimate Δ and VND from PET data, avoiding the limitations associated with linear regression. The performance of LEO was, naturally, dependent on the procedure used to estimate the covariance matrix, and the test-retest sample size. Given a test-retest sample size of at least 5, but preferably 10 individuals, LEO provides higher accuracy and precision than Lassen plot in the estimation of Δ and VND. We conclude that LEO is valuable in drug occupancy studies.

Bivariate Kumaraswamy distribution: properties and a new method to generate bivariate classes

In this paper, we introduce a bivariate Kumaraswamy (BVK) distribution whose marginals are Kumaraswamy distributions. The cumulative distribution function of this bivariate model has absolutely continuous and singular parts. Representations for the cumulative and density functions are presented and properties such as marginal and conditional distributions, product moments and conditional moments are obtained. We show that the BVK model can be obtained from the Marshall and Olkin survival copula and obtain a tail dependence measure. The estimation of the parameters by maximum likelihood is discussed and the Fisher information matrix is determined. We propose an EM algorithm to estimate the parameters. Some simulations are presented to verify the performance of the direct maximum-likelihood estimation and the proposed EM algorithm. We also present a method to generate bivariate distributions from our proposed BVK distribution. Furthermore, we introduce a BVK distribution which has only an absolutely continuous part and discuss some of its properties. Finally, a real data set is analysed for illustrative purposes.

Effective Single-Step Posttranscriptional Dynamics Allowing for a Direct Maximum Likelihood Estimation of Transcriptional Activity and the Quantification of Sources of Gene Expression Variability with an Illustration for the Hypoxia and TNFαRegulated Inflammatory Pathway

Data analysis methods for estimating promoter activity from gene reporter data frequently involve the reconstruction of the dynamics of unobserved species and numerical search algorithms for determining optimal model parameters. In contrast, we argue that posttranscriptional dynamics effectively behave like a singlestep stochastic process when gene expression variability is relatively low and, half-lives of the unobserved species are relatively small compared to characteristic observation time scales. In this case, by means of maximum likelihood estimators, for which analytical expressions exist, transcriptional activity of gene promoters can be estimated directly from observed gene reporter data without the need for numerical search algorithms and the reconstruction of unobserved variables. In addition, the model-based data analysis approach yields a single variable that measures the effective strength of the sources that give rise to gene expression variability. The approach is applied to conduct a model-based analysis of the inflammatory pathway under hypoxia condition and stimulation with tumor necrosis factor alpha in HEK293 cells.

Maximum likelihood estimation of direction of arrival using an acoustic vector-sensor

A vector-sensor consisting of a monopole sensor collocated with orthogonally oriented dipole sensors is used for direction of arrival (DOA) estimation in the presence of an isotropic noise-field or internal device noise. A maximum likelihood (ML) DOA estimator is derived and subsequently shown to be a special case of DOA estimation by means of a search for the direction of maximum steered response power (SRP). The problem of SRP maximization with respect to a vector-sensor can be solved with a computationally inexpensive algorithm. The ML estimator achieves asymptotic efficiency and thus outperforms existing estimators with respect to the mean square angular error (MSAE) measure. The beampattern associated with the ML estimator is shown to be identical to that used by the minimum power distortionless response beamformer for the purpose of signal enhancement.

Application of natural computing algorithms to maximum likelihood estimation of direction of arrival

This work presents a study of the performance of populational meta-heuristics belonging to the field of natural computing when applied to the problem of direction of arrival (DOA) estimation, as well as an overview of the literature about the use of such techniques in this problem. These heuristics offer a promising alternative to the conventional approaches in DOA estimation, as they search for the global optima of the maximum likelihood (ML) function in a framework characterized by an elegant balance between global exploration and local improvement, which are interesting features in the context of multimodal optimization, to which the ML-DOA estimation problem belongs. Thus, we shall analyze whether these algorithms are capable of implementing the ML estimator, i.e., finding the global optima of the ML function. In this work, we selected three representative natural computing algorithms to perform DOA estimation: differential evolution, clonal selection algorithm, and the particle swarm. Simulation results involving different scenarios confirm that these methods can reach the performance of the ML estimator, regardless of the number of sources and/or their nature. Moreover, the number of points evaluated by such methods is quite inferior to that associated with a grid search, which gives support to their application.

A GLM approach to step-stress accelerated life testing with interval censoring

In this paper, we present a statistical inference procedure for the step-stress accelerated life testing (SSALT) model with Weibull failure time distribution and interval censoring via the formulation of generalized linear model (GLM). The likelihood function of an interval censored SSALT is in general too complicated to obtain analytical results. However, by transforming the failure time to an exponential distribution and using a binomial random variable for failure counts occurred in inspection intervals, a GLM formulation with a complementary log–log link function can be constructed. The estimations of the regression coefficients used for the Weibull scale parameter are obtained through the iterative weighted least square (IWLS) method, and the shape parameter is updated by a direct maximum likelihood (ML) estimation. The confidence intervals for these parameters are estimated through bootstrapping. The application of the proposed GLM approach is demonstrated by an industrial example.

Fitting Fully Observed Recursive Mixed-process Models with cmp

At the heart of many econometric models are a linear function and a normal error. Examples include the classical small-sample linear regression model and the probit, ordered probit, multinomial probit, tobit, interval regression, and truncated-distribution regression models. Because the normal distribution has a natural multidimensional generalization, such models can be combined into mul-tiequation systems in which the errors share a multivariate normal distribution. The literature has historically focused on multistage procedures for fitting mixed models, which are more efficient computationally, if less so statistically, than maximum likelihood. Direct maximum likelihood estimation has been made more practical by faster computers and simulated likelihood methods for estimating higher-dimensional cumulative normal distributions. Such simulated likelihood methods include the Geweke–Hajivassiliou-Keane algorithm (Geweke, 1989, Econometrica 57: 1317–1339; Hajivassiliou and McFadden, 1998, Econometrica 66: 863–896; Keane, 1994, Econometrica 62: 95–116). Maximum likelihood also facilitates a generalization to switching, selection, and other models in which the number and types of equations vary by observation. The Stata command cmp fits seemingly unrelated regressions models of this broad family. Its estimator is also consistent for recursive systems in which all endogenous variables appear on the right-hand sides as observed. If all the equations are structural, then estimation is full-information maximum likelihood. If only the final stage or stages are structural, then estimation is limited-information maximum likelihood. cmp can mimic a score of built-in and user-written Stata commands. It is also appropriate for a panoply of models that previously were hard to estimate. Heteroskedasticity, however, can render cmp inconsistent. This article explains the theory and implementation of cmp and of a related Mata function, ghk2(), that implements the Geweke–Hajivassiliou–Keane algorithm.

Application of nonlinear filtering to credit risk

Merton’s model views equity as a call option on the asset of the firm. Thus the asset is partially observed through the equity. Then using nonlinear filtering an explicit expression for likelihood ratio for underlying parameters in terms of the nonlinear filter is obtained. As the evolution of the filter itself depends on the parameters in question, this does not permit direct maximum likelihood estimation, but does pave the way for the ‘Expectation–Maximization’ method for estimating parameters.

Efficient Estimation of a Narrow-Band Polynomial Phase Signal Impinging on a Sensor Array

<para xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> The parameters of interest of a polynomial phase signal observed by a sensor array include the direction of arrival and the polynomial coefficients. The direct maximum likelihood estimation of these parameters requires a nonlinear multidimensional search. In this paper, we present a two-step estimation approach. The estimation requires only a one-dimensional search in the direction of arrival space and involves a simple least squares solution for the polynomial coefficients. The efficiency of the estimates is corroborated by Monte Carlo simulations. </para>

Improving the threshold performance of maximum likelihood estimation of direction of arrival

We propose to improve the performance of some direction of arrival (DOA) estimators using array of sensors. We consider those maximum likelihood (ML) estimators that generate some DOA candidates and select one of them through an ML criterion. Our proposal modifies the candidate selection process substituting the traditional sample covariance matrix by a new one computed after filtering the received data with an optimum noise reduction filter. Simulation results indicate an improvement of the performance at low signal-to-noise ratios (SNR) and a considerable reduction of the threshold SNR. The computation of the new selection cost function implies in a small increase in the overall computational effort.

Estimating Fully Observed Recursive Mixed-Process Models with Cmp

At the heart of many econometric models is a linear function and a normal error. Examples include the classical small-sample linear regression model and the probit, ordered probit, multinomial probit, Tobit, interval regression, and truncated distribution regression models. Because the normal distribution has a natural multidimensional generalization, such models can be combined into multi-equation systems in which the errors share a multivariate normal distribution. The literature has historically focused on multi-stage procedures for estimating mixed models, which are more efficiently computationally, if less so statistically, than maximum likelihood (ML). But faster computers and simulated likelihood methods such as the Geweke, Hajivassiliou, and Keane (GHK) algorithm for estimating higher dimensional cumulative normal distributions have made direct ML estimation practical. ML also facilitates a generalization to switching, selection, and other models in which the number and types of equations vary by observation. The Stata module CMP fits Seemingly Unrelated Regressions (SUR) models of this broad family. Its estimator is also consistent for recursive systems in which all endogenous variables appear on the right-hand-sides as observed. If all the equations are structural, then estimation is full-information maximum likelihood (FIML). If only the final stage or stages are, then it is limited-information maximum likelihood (LIML). CMP can mimic a dozen built-in Stata commands and several user-written ones. It is also appropriate for a panoply of models previously hard to estimate. Heteroskedasticity, however, can render it inconsistent. This paper explains the theory and implementation of cmp and of a related Mata function, ghk2(), that implements the GHK algorithm.

A Simplified Framework for Using Multiple Imputation in Social Work Research

Missing data are nearly always a problem in research, and missing values represent a serious threat to the validity of inferences drawn from findings. Increasingly, social science researchers are turning to multiple imputation to handle missing data. Multiple imputation, in which missing values are replaced by values repeatedly drawn from conditional probability distributions, is an appropriate method for handling missing data when values are not missing completely at random. However, use of this method requires developing an imputation model from the observed data. This is typically a rigorous and time-consuming process. To encourage wider adoption of multiple imputation in social work research, a simple framework for designing imputation models is presented. The framework and its ability to generate unbiased estimates are demonstrated in a simulation study. KEY WORDS: missing data; multiple imputation; nonresponse ********** Missing data are ubiquitous in social research, and missingness or nonresponse can represent a threat to the validity of inferences because of undue effects on efficiency, power, and parameter bias (Shadish, Cook, & Campbell, 2002). Social work researchers are now addressing missing data in a more rigorous manner. Recently, Saunders et al. (2006) and Choi, Golder, Gillmore, and Morrison (2005) described important data imputation methods and dispelled misunderstandings regarding popular imputation methods, such as mean substitution. Recent advances in analytic methods, such as multiple imputation (MI), are taking hold in social work research. With MI, missing values are replaced with values repeatedly drawn from simulated conditional probability distributions (Schafer, 1997), thus creating multiple versions of the data set. Each version of the data set is analyzed according to the data analysis model, and the multiple results are combined into point estimates (Rubin, 1996). A critical task in MI is to devise an imputation model (Allison, 2002) or missing data model (Graham, Olchowski, & Gilreath, 2007), which involves specifying the measures that are putatively associated with the missing values. Although this process adds additional steps, the specification of an imputation model and the creation of multiple data sets can produce less-biased estimates in the presence of missing data across a wide variety of data analysis techniques (Schafer, 1997). Besides MI, there are many other methods for addressing missing data (Schafer, 1999; Schafer & Graham, 2002). An equally rigorous method known as direct or full information maximum likelihood (FIML) estimation can produce unbiased estimates and correct standard errors in the presence of missing data. When the number of imputations is sufficiently large, identical missing data models will produce the same estimates under MI and FIML (Graham et al., 2007). Unlike MI, FIML is limited to maximum likelihood analytic techniques and the missing data model must be included in the analysis model. Although we focus on MI, the steps we describe for developing an imputation model are equally appropriate for use in the missing data model for FIML. On the basis of the MI literature, this article describes a framework for developing an imputation model for use with any free or commercial software package that performs MI. BRIEF REVIEW OF MISSING DATA CONCEPTS Generally, both MI and a broad range of missing data issues have received ample attention in the applied literature (Graham et al., 2007). We briefly discuss the three types of distributions that describe the randomness of nonresponse given that this property has consequences for the development of an imputation model. For a discussion of general missing data concepts that are not critical to understanding our discussion of imputation model development, we refer readers to Schafer and Graham (2002). Distribution of Nonresponse The probability distribution of nonresponse--more frequently referred to as the missing data or nonresponse mechanism (Rubin, 1976)--is both an important factor in the decision to impute with MI and a context for the development of an imputation model. …

Exploring Social Mobility with Latent Trajectory Groups

SummaryWe present a new methodological approach to the study of social mobility. We use a latent class growth analysis framework to identify five qualitatively distinct social class trajectory groups between 1980 and 2000 for male respondents to the 1970 British Cohort Study. We model the antecedents of trajectory group membership via multinomial logistic regression. Non-response, which is a considerable problem in long-term panels and cohort studies, is handled via direct maximum likelihood estimation, which is consistent and efficient when data are missing at random. Our results suggest a combination of meritocratic and ascriptive influences on the probability of membership in the different trajectory groups.