Mean Integrated Squared Error Research Articles

Estimation of density functionals via cross‐validation

AbstractIn density estimation, the mean integrated squared error (MISE) is commonly used as a measure of performance. In that setting, the cross‐validation criterion provides an unbiased estimator of the MISE minus the integral of the squared density. Since the minimum MISE is known to converge to zero, this suggests that the minimum value of the cross‐validation criterion could be regarded as an estimator of minus the integrated squared density. This novel proposal presents the outstanding feature that, unlike all other existing estimators, it does not need the choice of any tuning parameter. Indeed, it is proved here that this approach results in a consistent and efficient estimator, with remarkable performance in practice. Moreover, apart from this base case, it is shown how several other problems on density functional estimation can be similarly handled using this new principle, thus demonstrating full potential for further applications.

Augmenting a Simulation Campaign for Hybrid Computer Model and Field Data Experiments

The Kennedy and O’Hagan (KOH) calibration framework uses coupled Gaussian processes (GPs) to meta-model an expensive simulator (first GP), tune its “knobs” (calibration inputs) to best match observations from a real physical/field experiment and correct for any modeling bias (second GP) when predicting under new field conditions (design inputs). There are well-established methods for placement of design inputs for data-efficient planning of a simulation campaign in isolation, that is, without field data: space-filling, or via criterion like minimum integrated mean-squared prediction error (IMSPE). Analogues within the coupled GP KOH framework are mostly absent from the literature. Here we derive a closed form IMSPE criterion for sequentially acquiring new simulator data for KOH. We illustrate how acquisitions space-fill in design space, but concentrate in calibration space. Closed form IMSPE precipitates a closed-form gradient for efficient numerical optimization. We demonstrate that our KOH-IMSPE strategy leads to a more efficient simulation campaign on benchmark problems, and conclude with a showcase on an application to equilibrium concentrations of rare earth elements for a liquid–liquid extraction reaction.

Phase transitions in nonparametric regressions

When the unknown regression function of a single variable is known to have derivatives up to the (γ+1)th order bounded in absolute values by a common constant everywhere or a.e. (i.e., (γ+1)th degree of smoothness), the minimax optimal rate of the mean integrated squared error (MISE) is stated as 1n2γ+22γ+3 in the literature. This paper shows that: (i) if n≤γ+12γ+3, the minimax optimal MISE rate is lognnlog(logn) and the optimal degree of smoothness to exploit is roughly maxlogn2loglogn,1; (ii) if n>γ+12γ+3, the minimax optimal MISE rate is 1n2γ+22γ+3 and the optimal degree of smoothness to exploit is γ+1.The fundamental contribution of this paper is a set of metric entropy bounds we develop for smooth function classes. Some of our bounds are original, and some of them improve and/or generalize the ones in the literature (e.g., (Kolmogorov and Tikhomirov, 1959)). Our metric entropy bounds allow us to show phase transitions in the minimax optimal MISE rates associated with some commonly seen smoothness classes as well as non-standard smoothness classes, and can also be of independent interest outside the nonparametric regression problems.

On the convergence rate of d-dimensional fourth-order beta polynomial kernels

This article focuses on formulating the Asymptotic Mean Integrated Squared Error (AMISE) scheme for d-dimensional fourth-order beta polynomial kernels in the context of kernel density estimation. The primary objective is to assess how this scheme influences the convergence rate, which directly impacts the speed at which the estimated density converges to the true density, contributing to bias reduction. The article employs AMISE as a metric to quantify the overall dissimilarity between the estimated density fˆ and the true density f . Quantitatively, the proposed convergence scheme is compared to existing ones by Deheuvels and Jones et al. across various scenarios of sample sizes and dimensionalities. The study’s findings provide compelling evidence that fourth-order beta polynomial kernels exhibit a significantly faster convergence rate compared to the rates documented in the literature. This accelerated convergence rate implies a substantial improvement in bias reduction capabilities. The results underscore the potential effectiveness of fourth-order beta polynomial kernels as a powerful tool for enhancing the accuracy of kernel density estimation tasks.

A new wavelet-based estimation of conditional density via block threshold method

A new wavelet-based estimator of the conditional density is investigated. The estimator is constructed by combining a special ratio technique and applying a non negative estimator to the density function in the denominator. We used a wavelet shrinkage technique to find an adaptive estimator for this problem. In particular, a block thresholding estimator is proposed, and we prove that it enjoys powerful mean integrated squared error properties over Besov balls. Moreover, it is shown that convergence rates for the mean integrated squared error (MISE) of the adaptive estimator are optimal under some mild assumptions. Finally, a numerical example has been considered to illustrate the performance of the estimator.

Adaptive Nadaraya-Watson kernel regression estimators utilizing some non-traditional and robust measures: a numerical application of British food data

In nonparametric regression research, estimation of regression function is a prime concern. Recently, researchers developed some modified Nadaraya-Watson (N-W) regression estimators utilizing robust mean, median and harmonic mean. In this paper, we propose to utilize the additive combination of non-traditional measures i.e. (Hodges-Lehmann, Mid-Range, Tri-Mean, Quartile-Deviation) with the robust minimum covariance determinant (MCD) scale estimator in (N-W) regression estimator. Utilizing these measures, we get some new versions of (N-W) regression estimator. We also attempted to derive the properties of the proposed versions, such as bias, variance, mean square error (MSE), and mean integrated square error (MISE). The proposed estimators are compared with some of the existing estimators available in literature through a simulation study, utilizing two artificial populations. We also incorporated real-life application by taking British food data set denoted as Engel95, and assess the predictive ability of nonparametric regression, based on proposed and existing N-W estimators.

A Parzen–Rosenblatt type density estimator for circular data: exact and asymptotic optimal bandwidths

. For the Parzen–Rosenblatt type density estimator for circular data, we prove the existence of a minimizer, h MISE ( f ; K , n ) , of its exact mean integrated squared error (MISE) and show that it is asymptotically equivalent to the bandwidth h AMISE ( f ; K , n ) that minimizes the leading terms of the MISE, together with the order of convergence of the relative error h AMISE ( f ; K , n ) / h MISE ( f ; K , n ) − 1 . Some small and moderate sample size comparisons between the two bandwidths are also presented when the underlying density is a mixture of von Mises densities.

Computationally Efficient Adaptive Design of Experiments for Global Metamodeling through Integrated Error Approximation and Multicriteria Search Strategies

Gaussian processes (GPs) are a popular technique for global metamodeling applications. Their objective in such settings is to establish an efficient and globally accurate approximation of the response surface of computationally expensive simulation models. When developing such GPs, the design of (simulation) experiments (DoE) plays an important role in reducing the required number of model runs for obtaining accurate approximations. Sequential (adaptive) selection of experiments can provide significant advantages, especially when the response surface is characterized by localized nonlinearities. Such adaptive DoE strategies for global metamodeling applications typically focus on minimizing the predictive GP variance, representing an exploration strategy, while recent developments have additionally considered the reduction of the GP bias obtained through cross validation, representing an exploitation strategy. While significant focus has been placed on the definition of appropriate adaptive DoE criteria, computational challenges still exist that limit the widespread adoption of adaptive DoE techniques—for example, related to the additional computational demand for identifying the optimal new experiment(s) or the necessity to establish proper schemes to combine exploration and exploitation strategies. To address these specific challenges, this research investigates two new adaptive DoE formulations. The first one focuses on the approximation of the popular integrated mean square error (IMSE) DoE criterion. The computationally demanding GP predictive variance update (after addition of each candidate experiment), required in the original IMSE formulation, is replaced by an approximation based on the current predictive variance and the domain of influence that surrounds each new experiment. The approximation is established through a parametric formulation that leverages the GP kernel to describe the aforementioned domain, with characteristics that are progressively calibrated across the GP training stages, to minimize the discrepancy between the actual and the approximated IMSE. The second formulation establishes a multicriteria search for simultaneously identifying multiple Pareto optimal experiments that balance exploration and exploitation objectives, replacing conventional strategies that establish a weighted combination of these objectives to promote a single DoE selection criterion.

Bootstrap bandwidth selection for the pair correlation function of inhomogeneous spatial point processes

This work focuses on kernel estimation of the pair correlation function (PCF) for inhomogeneous spatial point processes. We propose a bootstrap bandwidth selector based on minimizing the mean integrated squared error (MISE). The variance term is estimated by nonparametric bootstrap, and the bias by a plug-in approach using a pilot estimator of the PCF. Kernel estimators of the PCF also require a pilot estimator of the first-order intensity. We test the performance of the bandwidth selector and the role of the pilot intensity estimator in a simulation study. The bootstrap bandwidth selector is competitive with cross-validation procedures, but the contribution of the bandwidth parameter to the goodness-of-fit of the kernel PCF estimator is minor in comparison with that of the pilot intensity function. The data-based kernel intensity estimator leads to biased kernel PCF estimators, while both kernel and parametric covariate-based intensities provide accurate estimators of the PCF.

Bandwidth selection for nonparametric regression with errors-in-variables

We propose two novel bandwidth selection procedures for the nonparametric regression model with classical measurement error in the regressors. Each method evaluates the prediction errors of the regression using a second (density) deconvolution. The first approach uses a typical leave-one-out cross-validation criterion, while the second applies a bootstrap approach and the concept of out-of-bag prediction. We show the asymptotic validity of both procedures and compare them to the SIMEX method in a Monte Carlo study. As well as dramatically reducing computational cost, the methods proposed in this article lead to lower mean integrated squared error (MISE) compared to the current state-of-the-art.

Wavelet Estimation of a Density From Observations of Almost Periodically Correlated Process Under Positive Quadrant Dependence

In this paper, we construct a new wavelet estimator of density for the component of a finite mixture under positive quadrant dependence. Our sample is extracted from almost periodically correlated processes. To evaluate our estimator we will determine a convergence speed from an upper bound for the mean integrated squared error (MISE). Our result is compared to the independent case which provides an optimal convergence rate.

Methodology for nonparametric bias reduction in kernel regression estimation

Abstract In this paper, we propose and investigate two new kernel regression estimators based on a bias reduction transformation technique. We study the properties of these estimators and compare them with Nadaraya–Watson’s regression estimator and Slaoui’s (2016) regression estimator. It turns out that, with an adequate choice of the parameters of the two proposed estimators, the rate of convergence of two estimators will be faster than the two classical estimators, and the asymptotic MISE (mean integrated squared error) will be smaller than the two classical estimators. We corroborate these theoretical results through simulations and a real Malaria dataset.

Convergence Analysis of Stochastic Kriging-Assisted Simulation with Random Covariates

We consider performing simulation experiments in the presence of covariates. Here, covariates refer to some input information other than system designs to the simulation model that can also affect the system performance. To make decisions, decision makers need to know the covariate values of the problem. Traditionally in simulation-based decision making, simulation samples are collected after the covariate values are known; in contrast, as a new framework, simulation with covariates starts the simulation before the covariate values are revealed and collects samples on covariate values that might appear later. Then, when the covariate values are revealed, the collected simulation samples are directly used to predict the desired results. This framework significantly reduces the decision time compared with the traditional way of simulation. In this paper, we follow this framework and suppose there are a finite number of system designs. We adopt the metamodel of stochastic kriging (SK) and use it to predict the system performance of each design and the best design. The goal is to study how fast the prediction errors diminish with the number of covariate points sampled. This is a fundamental problem in simulation with covariates and helps quantify the relationship between the offline simulation efforts and the online prediction accuracy. Particularly, we adopt measures of the maximal integrated mean squared error (IMSE) and integrated probability of false selection (IPFS) for assessing errors of the system performance and the best design predictions. Then, we establish convergence rates for the two measures under mild conditions. Last, these convergence behaviors are illustrated numerically using test examples. History: Accepted by Bruno Tuffin, area editor for simulation. Funding: This work was supported in part by Singapore Ministry of Education Academic Research Funds [Tier 1 Grants R-155-000-201-114 and A-0004822-00-00], the City University of Hong Kong [Grants 7005269 and 7005568], and the National Natural Science Foundation of China [Grant 72091211]. Supplemental Material: The software that supports the findings of this study is available within the paper and its Supplemental Information ( https://pubsonline.informs.org/doi/suppl/10.1287/ijoc.2022.1263 ) as well as from the IJOC GitHub software repository ( https://github.com/INFORMSJoC/2021.0329 ) at ( http://dx.doi.org/10.5281/zenodo.7344997 ).

On a new higher order kernel for density estimation

Higher order kernels have been widely implemented for nonparametric function estimation, mainly due to their faster asymptotic rates of convergence and interpretability. This article constructs a novel higher order kernel of any pre‐specified even order by using Shannon's formula. The proposed kernels have a closed form expression and are easy to implement. We compare the constructed kernel with several popular higher order kernels in the context of density estimation. Further, we observe that the mean integrated squared error (MISE) of the density estimator corresponding to the proposed kernel is comparable with that of the popular kernels. The constructed kernel performs somewhat better for the Fejér‐de la Vallée Poussin and the ‐symmetric densities. The MISE of the corresponding kernel density estimator and a few related theoretical results are also presented.

Sequential nonparametric estimation of controlled multivariate regression

The article considers an adaptive sequential nonparametric estimation of a multivariate regression with assigned mean integrated squared error (MISE) and minimax mean stopping time when the estimator matches performance of an oracle knowing all nuisance parameters and functions. It is known that the problem has no solution if regression belongs to a Sobolev class of differentiable functions. What if an underlying regression is smoother, say, analytic? It is shown that in this case it is possible to match performance of the oracle. Furthermore, similar to the classical Stein solution for a parameter estimation, a two-stage sequential procedure solves the problem. The proposed regression estimator for the first stage, based on a sample with fixed sample size, is of interest on its own, and a thought-provoking environmental example of reducing potent greenhouse gas emission by an anaerobic digestion system is used to discuss a number of important topics for small samples.

Nonparametric bivariate density estimation for censored lifetimes

It is well known that estimation of a bivariate cumulative distribution function of a pair of right censored lifetimes presents challenges unparalleled to the univariate case where a product-limit Kaplan–Meyer’s methodology typically yields optimal estimation, and the literature on optimal estimation of the joint probability density is next to none. The paper, for the first time in the survival analysis literature, develops the theory and methodology of sharp minimax and adaptive nonparametric estimation of the joint density under the mean integrated squared error (MISE) criterion. The theory shows how an underlying joint density, together with the bivariate distribution of censoring variables, affect the estimation, and what and how may or may not be estimated in the presence of censoring. Practical example illustrates the problem.

Nonlinear wavelet-based estimation to spectral density for stationary non-Gaussian linear processes

Nonlinear wavelet-based estimators for spectral densities of non-Gaussian linear processes are considered. The convergence rates of mean integrated squared error (MISE) for those estimators over a large range of Besov function classes are derived, and it is shown that those rates are identical to minimax lower bounds in standard nonparametric regression model within a logarithmic term. Thus, those rates could be considered as nearly optimal. Therefore, the resulting wavelet-based estimators outperform traditional linear methods if the degree of smoothness of spectral densities varies considerably over the interval of interest, such as sharp spike, cusp, bump, etc., since linear estimators are not able to attain these rates. Unlike in classical nonparametric regression with Gaussian noise errors where thresholds are determined by normal distribution, we determine the thresholds based on a Bartlett type approximation of a quadratic form with dependent variables by its corresponding quadratic form with independent identically distributed (i.i.d.) random variables and Hanson-Wright inequality for quadratic forms in sub-gaussian random variables. The theory is illustrated with some numerical examples, and our simulation studies show that our proposed estimators are comparable to the current ones.

Using copula information in wavelet estimation of bivariate density function based on censorship observations

This article discusses the nonparametric estimation of a bivariate density function using copula information under right censoring. We propose an adaptive estimator based on wavelet methods and the formulae for the asymptotic mean integrated squared error(MISE) is used to get the near optimal rate on a large functional class of regular densities. In particular, the asymptotic formulae for MISE in the context of kernel density estimators is derived in the case of censoring. Finally, the consistency of the proposed estimators is established and its effectiveness is validated through a numerical simulations.

Maximum Likelihood Estimates and a survival function for fuzzy data for the Weibull-Pareto parameters

In this work, we discuss the estimation parameters and the reliability function for Weibull –Pareto distribution by maximum likelihood method. For that we used two iterative approximation techniques called “Newton-Raphson (NR)” and “Fisher’s Scoring (FS) method” when the available data is in the form of fuzzy numbers. The parameters’ and reliability function’s estimates are numerically contrasted using a Monte-Carlo simulation study in terms of their mean square error (MSE), integrated mean square error (IMSE), and mean absolute proportional error (MAPE) criteria. The FS technique outperforms the NR method in simulations, according to the empirical data.

Short-term electric load prediction using transfer learning with interval estimate adjustment

Although we are currently in the era of big data, it is always challenging to obtain complete and large-scale data due to the information protection for users and enterprises. In most cases, only partial data can be obtained, which harms the machine learning-based prediction model's performance. To effectively improve load forecasting accuracy, we propose a CNN-GRU hybrid model with parameter-based transfer learning. By transferring the training model details from a model with an extensive dataset to another model trained with a smaller dataset, the performance and accuracy of a prediction model with a smaller dataset can be improved. After that, we use the solve-the-equation (STE) method to estimate the bandwidth of data distribution by minimizing the mean integrated square error (MISE), this will in conjunction with prediction results provides a load prediction interval, which can provide the future load curve's fluctuation range at a specific confidence interval (CI). To verify the proposed method's effectiveness, we use residential data from the United States and commercial data from South Korea for predictive analysis. The experimental results show that the load forecast method with interval estimate adjustment proposed in this paper can effectively improve the accuracy and reliability of load forecasting under the scarcity of data.