Likelihood-based Estimator Research Articles

Robust likelihood estimation of dynamic panel data models

We develop likelihood-based estimators for autoregressive panel data models that are consistent in the presence of time series heteroskedasticity. Bias-corrected conditional score estimators, random effects maximum likelihood in levels and first differences, and estimators that impose mean stationarity are considered for general autoregressive models with individual effects. We investigate identification under unit roots, and show that random effects estimation in levels may achieve substantial efficiency gains relative to estimation from data in differences. In an empirical application, we find evidence against unit roots in individual earnings processes from the Panel Study of Income Dynamics and the Spanish section of the European Community Household Panel.

Estimating duration distribution aided by auxiliary longitudinal measures in presence of missing time origin.

Understanding the distribution of an event duration time is essential in many studies. The exact time to the event is often unavailable, and thus so is the full event duration. By linking relevant longitudinal measures to the event duration, we propose to estimate the duration distribution via the first-hitting-time model (e.g. Lee and Whitmore in Stat Sci 21(4):501–513, 2006). The longitudinal measures are assumed to follow a Wiener process with random drift. We apply a variant of the MCEM algorithm to compute likelihood-based estimators of the parameters in the longitudinal process model. This allows us to adapt the well-known empirical distribution function to estimate the duration distribution in the presence of missing time origin. Estimators with smooth realizations can then be obtained by conventional smoothing techniques. We establish the consistency and weak convergence of the proposed distribution estimator and present its variance estimation. We use a collection of wildland fire records from Alberta, Canada to motivate and illustrate the proposed approach. The finite-sample performance of the proposed estimator is examined by simulation. Viewing the available data as interval-censored times, we show that the proposed estimator can be more efficient than the well-established Turnbull estimator, an alternative that is often applied in such situations. Supplementary InformationThe online version contains supplementary material available at 10.1007/s10985-021-09520-w.

Bootstrap inference for Hawkes and general point processes

Inference and testing in general point process models such as the Hawkes model is predominantly based on asymptotic approximations for likelihoodbased estimators and tests, as originally developed in Ogata (1978). As an alternative, and to improve finite sample performance, this paper considers bootstrap-based inference for interval estimation and testing. Specifically, for a wide class of point process models we consider a novel bootstrap scheme labeled 'fixed intensity bootstrap' (FIB), where the conditional intensity is kept fixed across bootstrap repetitions. The FIB, which is very simple to implement and fast in practice, naturally extends previous ideas from the bootstrap literature on time series in discrete time, where the so-called 'fixed design' and 'fixed volatility' bootstrap schemes have shown to be particularly useful and effective. We compare the FIB with the classic recursive bootstrap, which is here labeled 'recursive intensity bootstrap' (RIB). In RIB algorithms, the intensity is stochastic in the bootstrap world and implementation of the bootstrap is more involved, due to its sequential structure. For both bootstrap schemes, no asymptotic theory is available; we therefore provide here a new bootstrap (asymptotic) theory, which allows to assess bootstrap validity. We also introduce novel 'nonparametric' FIB and RIB schemes, which are based on resampling time-changed transformations of the original waiting times. We show effectiveness of the different bootstrap schemes in finite samples through a set of detailed Monte Carlo experiments. As far as we are aware, this is the first detailed Monte Carlo study of bootstrap implementations for Hawkes-type processes. Finally, in order to illustrate, we provide applications of the bootstrap to both financial data and social media data.

Diffusion copulas: Identification and estimation

We propose a new semiparametric approach for modelling nonlinear univariate diffusions, where the observed process is a nonparametric transformation of an underlying parametric diffusion (UPD). This modelling strategy yields a general class of semiparametric Markov diffusion models with parametric dynamic copulas and nonparametric marginal distributions. We provide primitive conditions for the identification of the UPD parameters together with the unknown transformations from discrete samples. Likelihood-based estimators of both parametric and nonparametric components are developed and we analyse their asymptotic properties. Kernel-based drift and diffusion estimators are also proposed and shown to be normally distributed in large samples. A simulation study investigates the finite sample performance of our estimators in the context of modelling US short-term interest rates. We also present a simple application of the proposed method for modelling the CBOE volatility index data.

Population empirical likelihood estimation in dual frame surveys

Dual frame surveys are a device to reduce the costs derived from data collection in surveys and improve coverage for the whole target population. Since their introduction, in the 1960‘s, dual frame surveys have gained much attention and several estimators have been formulated based on a number of different approaches. In this work, we propose new dual frame estimators based on the population empirical likelihood method originally proposed by Chen and Kim (Stat Sin 24:335–355, 2014) and using both the dual and the single frame approach. The extension of the proposed methodology to more than two frame surveys is also sketched. The performance of the proposed estimators in terms of relative bias and relative mean squared error is tested through simulation experiments. These experiments indicate that the proposed estimators yield better results than other likelihood-based estimators proposed in the literature.

Blind Data Detection With Unknown Channel Coding

Blind data detection is widely investigated since it finds many crucial applications in the interference cancelation, cognitive radio, and military interception, etc. In this letter, we propose a blind iterative algorithm that simultaneously improves the performance of channel coding identification and data detection in the communication system with the unknown channel coding and the unknown fading channels. First, the likelihood-based channel estimator and the soft detector and regenerator are adopted in an iterative manner, which estimates the unknown parameters and detects the information bits. Then, an average log-likelihood ratio (LLR) classifier is adopted to identify the unknown channel coding and determine the information bits. Numerical results show that the proposed algorithm outperforms the existing scheme in terms of correct identification and data detection.

A Bayesian hierarchical copula model

Dependent data with hierarchical structures arises commonly from a variety of application, and analysis of such data is often challenging due to the complexity in modeling dependence structures and the computation intensity. In this paper, we propose a Bayesian hierarchical copula model (BHCM) to accommodate hierarchical structures of dependent data, where the subject-level dependence is modeled by the copula-based model and the hierarchical structure is described using random dependence parameters. We introduce a layer-by-layer sampling scheme for conducting Bayesian inferences. Our proposed BHCM enjoys the flexibility of modeling various complex association structures, while retaining manageable computation. Extensive simulation studies show that our proposed estimators outperform conventional likelihood-based estimators in a variety of finite sample settings. We apply the BHCM to analyze the Vertebral Column dataset arising from UCI Machine Learning Repository.

The Autoregressive Linear Mixture Model: A Time-Series Model for an Instantaneous Mixture of Network Processes

Vector autoregressive models provide a simple generative model for multivariate, time-series data. The autoregressive coefficients of the vector autoregressive model describe a network process. However, in real-world applications such as macroeconomics or neuroimaging, time-series data arise not from isolated network processes but instead from the simultaneous occurrence of multiple network processes. Standard vector autoregressive models cannot provide insights about the underlying structure of such time-series data. In this work, we present the autoregressive linear mixture (ALM) model. The ALM proposes a decomposition of time-series data into co-occurring network processes that we call autoregressive components. We also present a non-convex likelihood-based estimator for fitting the ALM model and show that it can be solved using the proximal alternating linearized minimization (PALM) algorithm. We validate the ALM on both synthetic and real-world electroencephalography data, showing that we can disambiguate task-relevant autoregressive components that correspond with distinct network processes.

Empirical comparisons of abundance estimators for two sympatric carnivores using noninvasive genetic sampling

Abundance estimators are often evaluated with simulations, or by comparing estimates to populations of known size. Advances in noninvasive genetic sampling have fueled an increase in the use of noninvasive genetic sampling-based capture–recapture. However, when working with free-ranging populations of unknown size, managers often lack data necessary to select the appropriate estimator. This leads to uncertainty regarding how choice of estimator or sampling design influence estimates, and managers may select estimators based on funding or logistical constraints. Alternatively, comparing estimates from multiple estimators can provide managers with greater confidence in estimates, or highlight potential differences. We used noninvasive genetic sampling to estimate the abundance of free-ranging kit foxes Vulpes macrotis and coyotes Canis latrans. We generated estimates of abundance with two non-spatial likelihood-based estimators: 1) robust design Huggins capture–recapture models and 2) single-occasion capture with replacement (CAPWIRE) models. We compared these with recently published estimates derived from spatially explicit capture–recapture (SECR) models. For both species, estimates from Huggins models were generally lower than those from SECR. Abundance estimates from CAPWIRE, which was developed specifically for noninvasive genetic sampling and generates estimates from a single sampling occasion, tended to be biased low with high precision. Our results suggest that choice of estimator and sampling design can significantly influence estimates, and that the relationship between estimators varied between species. Our results further suggest that single-occasion sampling often employed with CAPWIRE abundance estimation may produce biased results and be inappropriate for species requiring dispersed sampling strategies.

Best linear inverse probability weighted estimation for two-phase designs and missing covariate regression.

The inverse probability weighted estimator is often applied to two-phase designs and regression with missing covariates. Inverse probability weighted estimators typically are less efficient than likelihood-based estimators but, in general, are more robust against model misspecification. In this paper, we propose a best linear inverse probability weighted estimator for two-phase designs and missing covariate regression. Our proposed estimator is the projection of the SIPW onto the orthogonal complement of the score space based on a working regression model of the observed covariate data. The efficiency gain is from the use of the association between the outcome variable and the available covariates, which is the working regression model. One advantage of the proposed estimator is that there is no need to calculate the augmented term of the augmented weighted estimator. The estimator can be applied to general missing data problems or two-phase design studies in which the second phase data are obtained in a subcohort. The method can also be applied to secondary trait case-control genetic association studies. The asymptotic distribution is derived, and the finite sample performance of the proposed estimator is examined via extensive simulation studies. The methods are applied to a bladder cancer case-control study.

Joint Transmit and Receive Filter Optimization for Sub-Nyquist Delay-Doppler Estimation

In this article, a framework is presented for the joint optimization of the analog transmit and receive filter with respect to a parameter estimation problem. At the receiver, conventional signal processing systems restrict the two-sided bandwidth of the analog pre-filter $B$ to the rate of the analog-to-digital converter $f_s$ to comply with the well-known Nyquist-Shannon sampling theorem. In contrast, here we consider a transceiver that by design violates the common paradigm $B\leq f_s$. To this end, at the receiver, we allow for a higher pre-filter bandwidth $B>f_s$ and study the achievable parameter estimation accuracy under a fixed sampling rate when the transmit and receive filter are jointly optimized with respect to the Bayesian Cram\'{e}r-Rao lower bound. For the case of delay-Doppler estimation, we propose to approximate the required Fisher information matrix and solve the transceiver design problem by an alternating optimization algorithm. The presented approach allows us to explore the Pareto-optimal region spanned by transmit and receive filters which are favorable under a weighted mean squared error criterion. We also discuss the computational complexity of the obtained transceiver design by visualizing the resulting ambiguity function. Finally, we verify the performance of the optimized designs by Monte-Carlo simulations of a likelihood-based estimator.

Supervised dimension reduction for ordinal predictors

In applications involving ordinal predictors, common approaches to reduce dimensionality are either extensions of unsupervised techniques such as principal component analysis, or variable selection procedures that rely on modeling the regression function. A supervised dimension reduction method tailored to ordered categorical predictors is introduced which uses a model-based dimension reduction approach, inspired by extending sufficient dimension reductions to the context of latent Gaussian variables. The reduction is chosen without modeling the response as a function of the predictors and does not impose any distributional assumption on the response or on the response given the predictors. A likelihood-based estimator of the reduction is derived and an iterative expectation–maximization type algorithm is proposed to alleviate the computational load and thus make the method more practical. A regularized estimator, which simultaneously achieves variable selection and dimension reduction, is also presented. Performance of the proposed method is evaluated through simulations and a real data example for socioeconomic index construction, comparing favorably to widespread use techniques.

Nonparametric Maximum Likelihood Estimators of Time-Dependent Accuracy Measures for Survival Outcome Under Two-Stage Sampling Designs

ABSTRACTLarge prospective cohort studies of rare chronic diseases require thoughtful planning of study designs, especially for biomarker studies when measurements are based on stored tissue or blood specimens. Two-phase designs, including nested case–control and case-cohort sampling designs, provide cost-effective strategies for conducting biomarker evaluation studies.Existing literature for biomarker assessment under two-phase designs largely focuses on simple inverse probability weighting (IPW) estimators. Drawing on recent theoretical development on the maximum likelihood estimators for relative risk parameters in two-phase studies, we propose nonparametric maximum likelihood-based estimators to evaluate the accuracy and predictiveness of a risk prediction biomarker under both types of two-phase designs. In addition, hybrid estimators that combine IPW estimators and maximum likelihood estimation procedure are proposed to improve efficiency and alleviate computational burden. We derive large sample properties of proposed estimators and evaluate their finite sample performance using numerical studies. We illustrate new procedures using a two-phase biomarker study aiming to evaluate the accuracy of a novel biomarker, des-γ-carboxy prothrombin, for early detection of hepatocellular carcinoma. Supplementary materials for this article are available online.

Proportion estimation in ranked set sampling in the presence of tie information

Ranked set sampling (RSS) is a statistical technique that uses auxiliary ranking information of unmeasured sample units in an attempt to select a more representative sample that provides better estimation of population parameters than simple random sampling. However, the use of RSS can be hampered by the fact that a complete ranking of units in each set must be specified when implementing RSS. Recently, to allow ties declared as needed, Frey (Environ Ecol Stat 19(3):309–326, 2012) proposed a modification of RSS, which is to simply break ties at random so that a standard ranked set sample is obtained, and meanwhile record the tie structure for use in estimation. Under this RSS variation, several mean estimators were developed and their performance was compared via simulation, with focus on continuous outcome variables. We extend the work of Frey (2012) to binary outcomes and investigate three nonparametric and three likelihood-based proportion estimators (with/without utilizing tie information), among which four are directly extended from existing estimators and the other two are novel. Under different tie-generating mechanisms, we compare the performance of these estimators and draw conclusions based on both simulation and a data example about breast cancer prevalence. Suggestions are made about the choice of the proportion estimator in general.

Parameter Estimation With Out-of-Sample Objective

We study parameter estimation from the sample X, when the objective is to maximize the expected value of a criterion function, Q, for a distinct sample, Y. This is the situation that arises when a model is estimated for the purpose of describing other data than those used for estimation. The motivated for much estimation has this form, with forecasting problems being a prime example. A natural estimator is the innate estimator that maximizes Q(X;\theta.) wrt. \theta. While the innate estimator has certain advantages, we show that the asymptotically efficient estimator is defined from a likelihood function in conjunction with Q. The likelihood-based estimator is, however, fragile, as misspecification is harmful in two ways. First, the likelihood-based estimator may be inefficient under misspecification. Second, and more importantly, the likelihood approach requires a parameter transformation that depends on the truth, causing an improper mapping to be used under misspecification. The theoretical results are illustrated with two applications comprising asymmetric loss and multi-step forecasting, respectively.

Cooperative Spectrum Sensing With M-Ary Quantized Data in Cognitive Radio Networks Under SSDF Attacks

In this paper, we address the challenging and important cooperative spectrum sensing (CSS) problem with M-ary quantized data under spectrum sensing data falsification (SSDF) attacks. We introduce a probabilistic SSDF attack model to characterize the attacks by a malicious secondary user (SU). We analyze the attack behavior and derive the condition to nullify the detection capability of the fusion center (FC). To defend against the SSDF attacks, we propose a novel attack-proof CSS scheme with M-ary quantized data, mainly including a malicious SU identification method and an adaptive linear combination rule. By using the malicious SU identification approach, FC identifies malicious SUs and removes them from the data fusion process. The adaptive linear combination rule adjusts the weighted coefficients with the distribution parameter sets of identified normal SUs estimated using a maximum likelihood-based estimator. FC performs the spectrum sensing process with M-ary quantized data from the identified normal SUs. Comprehensive evaluation is conducted. Evaluation results show that the proposed malicious SU identification method can remove malicious SUs successfully and the proposed CSS scheme with M-ary quantized data is robust against the SSDF attacks.

Covariance Matrix Estimation Under Degeneracy for Complex Elliptically Symmetric Distributions

In several adaptive algorithms, the data covariance matrix must be frequently estimated, particularly in modern wireless systems where, due to high mobility, signal statistics change in time. When the underlying distribution is degenerate, likelihood-based estimators of the covariance matrix and/or scatter matrix may not exist for non-Gaussian signals. A well-known class of non-Gaussian distributions is the family of complex elliptically symmetric (CES) distributions, which includes several well-known distributions as special cases. In this paper, we give some results on the degeneracy of CES distributions and derive the maximum likelihood (ML) estimator and $M$ -estimators of the scatter matrix for degenerate CES vectors. Estimation with insufficient data is addressed by a regularization approach. We also formulate the Cramer–Rao lower bound (CRLB) on the elements of the scatter matrix for degenerate CES vectors. Our results coincide with those of the nondegenerate CES vectors whenever the scatter matrix is full rank. Numerical results are presented to justify the efficiency of the proposed method.

An alternative empirical likelihood method in missing response problems and causal inference.

Missing responses are common problems in medical, social, and economic studies. When responses are missing at random, a complete case data analysis may result in biases. A popular debias method is inverse probability weighting proposed by Horvitz and Thompson. To improve efficiency, Robins et al. proposed an augmented inverse probability weighting method. The augmented inverse probability weighting estimator has a double-robustness property and achieves the semiparametric efficiency lower bound when the regression model and propensity score model are both correctly specified. In this paper, we introduce an empirical likelihood-based estimator as an alternative to Qin and Zhang (2007). Our proposed estimator is also doubly robust and locally efficient. Simulation results show that the proposed estimator has better performance when the propensity score is correctly modeled. Moreover, the proposed method can be applied in the estimation of average treatment effect in observational causal inferences. Finally, we apply our method to an observational study of smoking, using data from the Cardiovascular Outcomes in Renal Atherosclerotic Lesions clinical trial. Copyright © 2016 John Wiley & Sons, Ltd.

SSRs, SNPs and DArTs comparison on estimation of relatedness and genetic parameters’ precision from a small half-sib sample population of Eucalyptus grandis

Simple sequence repeats (SSR) are the most widely used molecular markers for relatedness inference due to their multi-allelic nature and high informativeness. However, there is a growing trend toward using high-throughput and inter-specific transferable single-nucleotide polymorphisms (SNP) and Diversity Arrays Technology (DArT) in forest genetics owing to their wide genome coverage. We compared the efficiency of 15 SSRs, 181 SNPs and 2816 DArTs to estimate the relatedness coefficients, and their effects on genetic parameters’ precision, in a relatively small data set of an open-pollinated progeny trial of Eucalyptus grandis (Hill ex Maiden) with limited relationship from the pedigree. Both simulations and real data of Eucalyptus grandis were used to study the statistical performance of three relatedness estimators based on co-dominant markers. Relatedness estimates in pairs of individuals belonging to the same family (related) were higher for DArTs than for SNPs and SSRs. DArTs performed better compared to SSRs and SNPs in estimated relatedness coefficients in pairs of individuals belonging to different families (unrelated) and showed higher ability to discriminate unrelated from related individuals. The likelihood-based estimator exhibited the lowest root mean squared error (RMSE); however, the differences in RMSE among the three estimators studied were small. For the growth traits, heritability estimates based on SNPs yielded, on average, smaller standard errors compared to those based on SSRs and DArTs. Estimated relatedness in the realized relationship matrix and heritabilities can be accurately inferred from co-dominant or sufficiently dense dominant markers in a relatively small E. grandis data set with shallow pedigree.

A structured Dirichlet mixture model for compositional data: inferential and applicative issues

The flexible Dirichlet (FD) distribution (Ongaro and Migliorati in J. Multvar. Anal. 114: 412---426, 2013) makes it possible to preserve many theoretical properties of the Dirichlet one, without inheriting its lack of flexibility in modeling the various independence concepts appropriate for compositional data, i.e. data representing vectors of proportions. In this paper we tackle the potential of the FD from an inferential and applicative viewpoint. In this regard, the key feature appears to be the special structure defining its Dirichlet mixture representation. This structure determines a simple and clearly interpretable differentiation among mixture components which can capture the main features of a large variety of data sets. Furthermore, it allows a substantially greater flexibility than the Dirichlet, including both unimodality and a varying number of modes. Very importantly, this increased flexibility is obtained without sharing many of the inferential difficulties typical of general mixtures. Indeed, the FD displays the identifiability and likelihood behavior proper to common (non-mixture) models. Moreover, thanks to a novel non random initialization based on the special FD mixture structure, an efficient and sound estimation procedure can be devised which suitably combines EM-types algorithms. Reliable complete-data likelihood-based estimators for standard errors can be provided as well.