Mean Squared Prediction Error Estimator Research Articles

Benchmarking the Observed Best Predictor

We propose two methods for benchmarking the observed best predictor (OBP; Jiang et al.[ 1 ]) in small area estimation under the Fay-Herriot model. Furthermore, we propose two methods for estimating the mean squared prediction error (MSPE) of the benchmarked OBP. Theoretical and empirical properties of the benchmarked OBP as well as its MSPE estimators are studied. A real-data example is considered. AMS Subject Classification: 62J05

An empirical comparison of various MSPE estimators and associated prediction intervals for small area means

Small area estimation is a method for prediction involving subpopulation of interest. A standard measure of uncertainty for the prediction is the mean squared prediction error (MSPE). Various predictors and MSPE estimators have been proposed based on different principles over past decades. Despite various existing studies on performance of individual MSPE estimators, a comprehensive comparison of different MSPE estimators in terms of finite sample performance, especially for several recently proposed MSPE estimators, has not been documented. To this end, we carry out a study that systematically compare the existing MSPE estimators regarding their finite sample performance and computational speed. Performance of prediction intervals is also considered, which involves both the predictor and the associated MSPE estimator. The study covers two main types of small area models, namely, the area-level (Fay-Herriot) model and the unit-level (nested-error regression) model. Illustrative examples are provided.

Estimation of mean squared prediction error of empirically spatial predictor of small area means under a linear mixed model

Policy decisions regarding allocation of resources to subgroups in a population, called small areas, are based on reliable predictors of their underlying parameters. However, the information is collected at a different scale than these subgroups. Hence, we need to predict characteristics of the subgroups based on the coarser scale data. In view of this, there is a growing demand for reliable small area predictors by borrowing information from other related sources. For this purpose, mixed models have been commonly used in small area estimation assuming independent small areas. There are many situations, however, that the small area parameters are related to their locations. For instance, it is an interest of policy makers (and public) to know the spatial pattern of a chronic disease (e.g., asthma) to identify small areas with high risk of disease for possible preventions. In this paper, we propose small area models in the class of spatial linear mixed models to be able to predict small area parameters and also to obtain corresponding mean squared prediction error (MSPE). We also provide unbiased estimators of MSPE of small area predictors using Taylor series expansion and parametric bootstrap methods. In our simulations, we show that our MSPE estimators using Taylor expansion and parametric bootstrap perform very well in terms of precision of small area predictors. Performance of our proposed approach is also evaluated through a real application of physician visits for Total Respiratory Morbidity conditions in Manitoba, Canada.

Sumca: Simple, Unified, Monte-Carlo-assisted Approach to Second-Order Unbiased Mean-Squared Prediction Error Estimation

SummaryWe propose a simple, unified, Monte-Carlo-assisted approach (called ‘Sumca’) to second-order unbiased estimation of the mean-squared prediction error (MSPE) of a small area predictor. The MSPE estimator proposed is easy to derive, has a simple expression and applies to a broad range of predictors that include the traditional empirical best linear unbiased predictor, empirical best predictor and post-model-selection empirical best linear unbiased predictor and empirical best predictor as special cases. Furthermore, the leading term of the MSPE estimator proposed is guaranteed positive; the lower order term corresponds to a bias correction, which can be evaluated via a Monte Carlo method. The computational burden for the Monte Carlo evaluation is much less, compared with other Monte-Carlo-based methods that have been used for producing second-order unbiased MSPE estimators, such as the double bootstrap and Monte Carlo jackknife. The Sumca estimator also has a nice stability feature. Theoretical and empirical results demonstrate properties and advantages of the Sumca estimator.

A new classified mixed model predictor

We develop a new method for classified mixed model prediction (CMMP). The original CMMP method (Jiang et al., 2018b) does not incorporate covariate information in matching the class between the new observations and the training data. As a result, the method may not outperform the mixed model prediction (MMP) method in terms of predictive performance. The new CMMP method that we develop utilizes covariate information, and therefore is more accurate in terms of the matching. We show that the new CMMP method outperforms the MMP in terms of the predictive performance. Furthermore, we develop a second-order unbiased estimator of the mean squared prediction error (MSPE) for the new CMMP, which was previously not available for the original CMMP. Theoretical and empirical properties of the proposed new CMMP method as well as the MSPE estimator are studied. A real data application is considered.

A class of general adjusted maximum likelihood methods for desirable mean squared error estimation of EBLUP under the Fay–Herriot small area model

The empirical best linear unbiased prediction (EBLUP) estimator is utilized for efficient inference in various research areas, especially for small-area estimation. In order to measure its uncertainty, we generally need to estimate its mean squared prediction error (MSE). Ideally, an EBLUP-based method should not only provide a second-order unbiased estimator of MSE of EBLUP but also maintain strict positivity in estimators of both model variance parameter and MSE of EBLUP. Fortunately, the MSE estimators proposed in Yoshimori and Lahiri (2014) and Hirose and Lahiri (2018) achieve the three desired properties simultaneously. As far as we know, no other MSE estimator does so.In this paper, we therefore seek an adequate class of general adjusted maximum-likelihood methods that simultaneously achieve the three desired properties of MSE estimation. To establish that the investigated class does so, we reveal the relationship between the general adjusted maximum-likelihood method for the model variance parameter and the general functional form of the second-order unbiased MSE estimator, maintaining strict positivity. We also compare the performance of several MSE estimators in our investigated class and others through a Monte Carlo simulation study. The results show that the MSE estimators in our investigated class perform better than those in others.

Small area estimation using survey weights under a nested error linear regression model with structural measurement error

Previously, the nested error linear regression models using survey weights have been studied in small area estimation to obtain efficient model-based and design-consistent estimators of small area means. In particular, the pseudo-empirical Bayes (PEB) using survey weights has received a lot of attention and is being used in statistical agencies. The covariates in these nested error linear regression models are not subject to measurement errors. However, there are many situations that the covariates are subject to measurement errors. In this paper, we develop a nested error linear regression model with an area-level covariate subject to structural measurement error. In particular, we propose a PEB estimator to estimate small area means. This estimator borrows strength across areas through the model and makes use of the survey weights to preserve the design consistency as the area sample size increases. We also employ a parametric bootstrap approach to estimate the mean squared prediction error (MSPE) of the PEB predictor. Finally, we report the results of a simulation study on the performance of our PEB predictor and associated bootstrap MSPE estimator.

SMALL AREA ESTIMATION USING SURVEY WEIGHTS WITH FUNCTIONAL MEASUREMENT ERROR IN THE COVARIATE

Summary Nested error linear regression models using survey weights have been studied in small area estimation to obtain efficient model-based and design-consistent estimators of small area means. The covariates in these nested error linear regression models are not subject to measurement errors. In practical applications, however, there are many situations in which the covariates are subject to measurement errors. In this paper, we develop a nested error linear regression model with an area-level covariate subject to functional measurement error. In particular, we propose a pseudo-empirical Bayes (PEB) predictor to estimate small area means. This predictor borrows strength across areas through the model and makes use of the survey weights to preserve the design consistency as the area sample size increases. We also employ a jackknife method to estimate the mean squared prediction error (MSPE) of the PEB predictor. Finally, we report the results of a simulation study on the performance of our PEB predictor and associated jackknife MSPE estimator.

On the Estimation of Mean Squared Prediction Error in Small Area Estimation

We critically examine the exact mean squared prediction error (MSPE) of an empirical best linear unbiased predictor (EBLUP) for certain special cases of the well-known Fay-Herriot model widely used in small area estimation. This study is useful in understanding the impact of the neglected higher order terms in standard second-order aymptotics in small area estimation problems. We argue that in certain cases the naive MSPE estimator, which is generally considered to have a serious underestimation problem as it ignores the uncertainty due to estimation of the variance component, may be adequate and may even overestimate the true MSPE. We propose a new weighted jackknife MSPE estimator as an alternative to the Taylor linearization method. Our proposed method is equivalent to other second-order unbiased MSPE estimators in terms of the standard second-order asymptotics. However, in our simulation study the proposed jackknife MSPE estimators appear to be better than the rival Taylor linearization method in terms of bias, especially when the regularity conditions that govern the second-order unbiasedness property are violated. We observe that for a special case of the Fay-Herriot model, our jackknife MSPE estimator is equivalent, in a higher order asymptotic sense, to a hierarchical Bayes solution under at priors on the hyperparameters.

Resampling-based bias-corrected time series prediction

In this paper, we consider estimation of the mean squared prediction error (MSPE) of the best linear predictor of (possibly) nonlinear functions of finitely many future observations in a stationary time series. We develop a resampling methodology for estimating the MSPE when the unknown parameters in the best linear predictor are estimated. Further, we propose a bias corrected MSPE estimator based on the bootstrap and establish its second order accuracy. Finite sample properties of the method are investigated through a simulation study.

Pseudo-empirical Bayes estimation of small area means under a nested error linear regression model with functional measurement errors

Small area estimation is studied under a nested error linear regression model with area level covariate subject to measurement error. Ghosh and Sinha (2007) obtained a pseudo-Bayes (PB) predictor of a small area mean and a corresponding pseudo-empirical Bayes (PEB) predictor, using the sample means of the observed covariate values to estimate the true covariate values. In this paper, we first derive an efficient PB predictor by using all the available data to estimate true covariate values. We then obtain a corresponding PEB predictor and show that it is asymptotically “optimal”. In addition, we employ a jackknife method to estimate the mean squared prediction error (MSPE) of the PEB predictor. Finally, we report the results of a simulation study on the performance of our PEB predictor and associated jackknife MSPE estimator. Our results show that the proposed PEB predictor can lead to significant gain in efficiency over the previously proposed PEB predictor. Area level models are also studied.

Empirical Bayes Estimation of Small Area Means under a Nested Error Linear Regression Model with Measurement Errors in the Covariates

Abstract. Previously, small area estimation under a nested error linear regression model was studied with area level covariates subject to measurement error. However, the information on observed covariates was not used in finding the Bayes predictor of a small area mean. In this paper, we first derive the fully efficient Bayes predictor by utilizing all the available data. We then estimate the regression and variance component parameters in the model to get an empirical Bayes (EB) predictor and show that the EB predictor is asymptotically optimal. In addition, we employ the jackknife method to obtain an estimator of mean squared prediction error (MSPE) of the EB predictor. Finally, we report the results of a simulation study on the performance of our EB predictor and associated jackknife MSPE estimators. Our results show that the proposed EB predictor can lead to significant gain in efficiency over the previously proposed EB predictor.

On Mean Squared Prediction Error Estimation in Small Area Estimation Problems

For a general linear mixed normal model, a new linearized weighted jackknife method is proposed to estimate the mean squared prediction error (MSPE) of an empirical best linear unbiased predictor (EBLUP) of a general mixed effect. Different MSPE estimators are compared using a Monte Carlo simulation study.

Estimation of Finite Population Domain Means

In this article we introduce a general methodology for producing a model-assisted empirical best predictor (EBP) of a finite population domain mean using data from a complex survey. Our method improves on the commonly used design-consistent survey estimator by using a suitable mixed model. Such a model combines information from related sources, such as census and administrative data. Unlike a purely model-based EBP, the proposed model-assisted EBP converges in probability to the customary design-consistent estimator as the domain and sample sizes increase. The convergence in probability is shown to hold with respect to the sampling design, irrespective of the assumed mixed model, a property commonly known as design consistency. This property ensures robustness of the proposed predictor against possible model failures. In addition, the convergence in probability is shown to be valid with respect to the assumed mixed model (model consistency). A new mean squared prediction error (MSPE) estimator is proposed. Unlike earlier MSPE estimators, our MSPE estimator is second-order unbiased. Our simulation results demonstrate the robustness properties of our proposed model-assisted predictor and the usefulness of the second-order unbiased MSPE estimator.