Minimum Variance Estimation Research Articles

Estimation and testing for panel data partially linear single-index models with errors correlated in space and time

We consider a panel data partially linear single-index models (PDPLSIM) with errors correlated in space and time. A serially correlated error structure is adopted for the correlation in time. We propose using a semiparametric minimum average variance estimation (SMAVE) to obtain estimators for both the parameters and unknown link function. We not only establish an asymptotically normal distribution for the estimators of the parameters in the single index and the linear component of the model, but also obtain an asymptotically normal distribution for the nonparametric local linear estimator of the unknown link function. Then, a fitting of spatial and time-wise correlation structures is investigated. Based on the estimators, we propose a generalized F-type test method to deal with testing problems of index parameters of PDPLSIM with errors correlated in space and time. It is shown that under the null hypothesis, the proposed test statistic follows asymptotically a [Formula: see text]-distribution with the scale constant and degrees of freedom being independent of nuisance parameters or functions. Simulated studies and real data examples have been used to illustrate our proposed methodology.

Process noise covariance estimation via stochastic approximation

SummaryKalman filtering for linear systems is known to provide the minimum variance estimation error, under the assumption that the model dynamics is known. While many system identification tools are available for computing the system matrices from experimental data, estimating the statistics of the output and process noises is still an open problem. Correlation‐based approaches are very fast and sufficiently accurate, but there are typically restrictions on the number of noise covariance elements that can be estimated. On the other hand, maximum likelihood methods estimate all elements with high accuracy, but they are computationally expensive, and they require the use of external optimization solvers. In this paper, we propose an alternative solution, tailored for process noise covariance estimation and based on stochastic approximation and gradient‐free optimization, that provides a good trade‐off in terms of performance and computational load, and is also easy to implement. The effectiveness of the method as compared to the state of the art is shown on a number of recently proposed benchmark examples.

A Measurement of the Cosmic Microwave Background Lensing Potential and Power Spectrum from 500 deg2 of SPTpol Temperature and Polarization Data

Abstract We present a measurement of the cosmic microwave background lensing potential using 500 deg2 of 150 GHz data from the SPTpol receiver on the South Pole Telescope. The lensing potential is reconstructed with signal-to-noise per mode greater than unity at lensing multipoles L ≲ 250, using a quadratic estimator on a combination of cosmic microwave background temperature and polarization maps. We report measurements of the lensing potential power spectrum in the multipole range of 100 < L < 2000 from sets of temperature-only (T), polarization-only (POL), and minimum-variance (MV) estimators. We measure the lensing amplitude by taking the ratio of the measured spectrum to the expected spectrum from the best-fit Λ cold dark matter model to the Planck 2015 TT + low P + lensing data set. For the minimum-variance estimator, we find A MV = 0.944 ± 0.058 ( Stat . ) ± 0.025 ( Sys . ) ; restricting to only polarization data, we find A POL = 0.906 ± 0.090 ( Stat . ) ± 0.040 ( Sys . ) . Considering statistical uncertainties alone, this is the most precise polarization-only lensing amplitude constraint to date (10.1σ) and is more precise than our temperature-only constraint. We perform null tests and consistency checks and find no evidence for significant contamination.

Analysis of panel data partially linear single-index models with serially correlated errors

In this paper, we study a panel data partially linear single-index models (PDPLSIM) with serially correlated errors. A semiparametric minimum average variance estimation method was given to estimate both the unknown parameters and link function. The asymptotic properties of the resulting estimators are established. Then, the serially correlated error structure is investigated. Furthermore, a generalized F-type test method was proposed to check the index parameters. It is shown that under the null hypothesis the proposed test statistic follows asymptotically a χ2-distribution. A numerical simulation and a real data example show that the proposed methods are effective.

Recursive linearly constrained Wiener filter for robust multi-channel signal processing

This article introduces a new class of recursive linearly constrained minimum variance estimators (LCMVEs) that provides additional robustness to modeling errors. To achieve that robustness, a set of non-stationary linear constraints are added to the standard LCMVE that allow for a closed form solution that becomes appealing in sequential implementations of the estimator. Indeed, a key point of such recursive LCMVE is to be fully adaptive in the context of sequential estimation as it allows optional constraints addition that can be triggered by a preprocessing of each new observation or external information on the environment. This methodology has significance in the popular problem of linear regression among others. Particularly, this article considers the general class of partially coherent signal (PCS) sources, which encompasses the case of fully coherent signal (FCS) sources. The article derivates the recursive LCMVE for this type of problems and investigates, analytically and through simulations, its robustness against mismatches on linear discrete state-space models. Both errors on system matrices and noise statistics uncertainty are considered. An illustrative multi-channel array processing example is treated to support the discussion, where results in different model mismatched scenarios are provided with respect to the standard case with only FCS sources.

Classical and Bayesian Estimation of Stress-Strength Reliability from Generalized Inverted Exponential Distribution based on Upper Records

This paper deals with the problem of classical and Bayesian estimation of stress-strength reliability (R=P(X<Y)) based on upper record values from generalized inverted exponential distribution (GIED). Hassan {et al.} (2018) discussed the maximum likelihood estimator (MLE) and Bayes estimator of $R$ by considering that the scale parameter to be known for defined distribution while we consider the case when all the parameters of GIED are unknown. In the classical approach, we have discussed MLE and uniformly minimum variance estimator (UMVUE). In Bayesian approach, we have considered the Bays estimator of R by considering the squared error loss function. Further, based on upper records, we have considered the Asymptotic confidence interval based on MLE, Bayesian credible interval and bootstrap confidence interval for $R$. Finally, Monte Carlo simulations and real data applications are being carried out for comparing the performances of the estimators of R.

Unscented Kalman Filter-Based Unbiased Minimum-Variance Estimation for Nonlinear Systems With Unknown Inputs

This letter proposes an unscented Kalman filter (UKF)-based unbiased minimum-variance estimation (UMV) method for the nonlinear system with unknown inputs. By utilizing the statistical linearization, the nonlinear system and measurement functions are transformed into a “linear-like” regression form. The latter preserves the nonlinearity of the system and the measurement models. To this end, the unknown inputs can be estimated by the weighted least-squares. This “linear-like” regression form also allows us to resort to the UMV state estimation framework for the development of new nonlinear filter to handle unknown inputs. Specifically, two approaches have been developed: 1) given the estimated inputs, we derive a filter by minimizing the trace of the state error covariance matrix; 2) without input estimation, we derive the filter by minimizing the trace of the state error covariance matrix subject to a constraint imposed on the gain matrix. We prove that these two approaches provide the same results. Numerical results validate the effectiveness of the proposed method.

Portfolio construction using bootstrapping neural networks: evidence from global stock market

The study investigates the investment value of global stock markets by a portfolio construction method combined with bootstrapping neural network architecture. A residual sample will be generated from bootstrapping sample procedure and then incorporated into the estimation of the expected returns and the covariant matrix. The outputs are further processed by the traditional Markowitz optimization procedure. In order to examine the efficacy of the proposed approach, the illustrated case was compared with traditional Markowitz mean–variance analysis, as well as the James–Stein and minimum-variance estimators. From the empirical results, it indicated that this novel approach significantly outperforms most of benchmark models based on various risk-adjusted performance measures. It can be shown that this new approach has great promise for enhancing the estimation of the investment value by Markowitz mean–variance analysis in the global stock markets.

Composite likelihood methods: Rao-type tests based on composite minimum density power divergence estimator

This paper is aimed to present a robust extension of the classical Rao test statistic, in the context of composite likelihood ideas and methods. The Rao-type test statistics are defined on the basis of the composite minimum power divergence estimators instead of the composite maximum likelihood estimator. These Rao-type test statistics are used to test simple and composite null hypotheses. Their performance is evaluated in terms of a simulation study which concentrates to the robustness and the comparison of the Rao-type tests with the respective Wald-type tests considered in Castilla et al. (Entropy 20:18, 2018). The proposed here procedures are developed on the basis of the restricted composite minimum density power divergence estimators which are also discussed for the sake of completeness.

Full-sky lensing reconstruction of 21 cm intensity maps

ABSTRACT Weak gravitational lensing of the 21 cm radiation is expected to be an important cosmological probe for post-reionization physics. We investigate the reconstruction of the matter density perturbations using a quadratic minimum variance estimator. The next generation of line intensity mapping (LIM) surveys such as HIRAX and CHIME will cover a larger sky fraction, which requires one to account for the curvature in the sky. Thus, we extend the plane-parallel flat-sky formalism for lensing reconstruction to account for a full-sky survey using the spherical Fourier–Bessel (SFB) expansion. Using the HIRAX 21 cm survey as a basis, we make predictions for lensing-reconstruction noise in our formalism and compare our results with the predictions from the plane-parallel formalism. We find agreement with the plane-parallel noise power spectrum at small scales and a significant deviation at scales L ≲ ℓres − keqR, where R is the radius of the shell volume, keq is the wavenumber for matter–radiation equality, and ℓres is the angular resolution scale. Furthermore, we derive the SFB flat-sky reconstruction noise and compare it with the full-sky SFB case as well as the plane-parallel case, finding minor deviations from the full-sky noise due to sphericity. We also determine that, in the absence of non-Gaussian statistics of the intensity field but accounting for foregrounds, the signal-to-noise ratio for $C_\ell ^{\phi \phi }$ using our SFB estimator increases by over 100 per cent. This shows that accounting for the curved sky in LIM weak lensing will be crucial for large-scale cosmology.

On bootstrap consistency of MAVE for single index models

This paper concerns the bootstrap consistency of the minimum average variance estimation (MAVE) method for the single index model. This paper shows that the conditional wild bootstrap estimator of the parameter index shares the same asymptotic covariance of the original MAVE estimator. Thus, the asymptotic distribution can be accurately estimated by the proposed wild bootstrap method. As an application of this method, this paper proposes a conditional Wald type test for the parameter index. It will be shown by simulations that the conditional bootstrap based test is more powerful than the test based on the traditional plug-in covariance estimator. A real data analysis is also provided to demonstrate the effectiveness of the bootstrap method.

Unbiased steady minimum-variance estimation for systems with measurement-delay and unknown inputs

This paper considers the problem of simultaneously estimating the state and the unknown input for linear discrete-time systems with measurement delay. Firstly, the reorganized innovation analysis approach is applied to deal with measurement delay and the measurement delay model is converted into a measurement delay free model. A recursive filter where the estimation of the state and the input are interconnected is proposed. Then we utilize the innovation to obtain the unknown input estimator by least-squares estimation and the optimal state estimator is constructed by transforming into a standard Kalman filtering in terms of two Riccati equations with the same dimension as the state model. Further, the infinite horizon asymptotic stability of proposed filter is discussed. Finally we give a numerical example to show that our estimation approach is effective.

State and parameter estimation using extended Kitanidis Kalman filter

State estimation in presence of unknown parameter variations has received considerable attention in literature. The conventional approach is to model the unknown drifting parameters as random walk model and estimate them simultaneously with states by augmenting them with the states of the system. However, the estimates critically depend on the tuning of the random walk model. Kitanidis Kalman Filter (KKF) [1] is an unbiased minimum variance estimator for only the states in presence of unknown inputs for linear systems. KKF allows optimal estimates of states to be obtained in presence of unknown inputs by appropriately choosing the gain matrix during the state update step. In this work, we investigate the problem of state and parameter estimation for nonlinear systems using extended Kitanidis Kalman Filter (EKKF). EKKF enables state and parameter estimation separately. In particular, we extend the work of Ganesh et al. [2] by providing a maximum likelihood based moving window framework for parameter estimation based on the innovations generated by EKKF. Towards this end, we characterise the innovations generated by KKF and show that unlike Kalman filter where the innovations are white, the innovations generated in KKF are a coloured Gaussian stochastic process. We also present a recursive approach for computing the covariance kernel of the innovations. The proposed method is applied to a simulation and an experimental case study to demonstrate its utility. The results indicate that the proposed method can be a useful tool for state and parameter estimation of nonlinear systems.

Bayesian Filtering With Unknown Sensor Measurement Losses

This work studies the state estimation problem of a stochastic nonlinear system with unknown sensor measurement losses. If the estimator knows the sensor measurement losses of a linear Gaussian system, the minimum variance estimate is easily computed by the celebrated intermittent Kalman filter (IKF). However, this will no longer be the case when the measurement losses are unknown and/or the system is nonlinear or non-Gaussian. By exploiting the binary property of the measurement loss process and the IKF, we design three suboptimal filters for the state estimation, i.e., BKF-I, BKF-II and RBPF. The BKF-I is based on the MAP estimator of the measurement loss process and the BKF-II is derived by estimating the conditional loss probability. The RBPF is a particle filter based algorithm which marginalizes out the loss process to increase the efficiency of particles. All the proposed filters can be easily implemented in recursive forms. Finally, a linear system, a target tracking system and a quadrotor's path control problem are included to illustrate their effectiveness, and show the tradeoff between computational complexity and estimation accuracy of the proposed filters.

On the equivalence between the unbiased minimum-variance estimation and the infinity augmented Kalman filter

ABSTRACT In this paper, the augment Kalman filter (AKF) approach is discussed under the infinity condition. It is proved that the AKF in infinity condition is equivalent to the minimum-variance unbiased (MVU) estimation proposed in Gillijns & De Moor [(2007). Unbiased minimum-variance input and state estimation for linear discrete-time systems. Automatica, 43(1), 111–116]. This result can be regarded as a new contribution in the field of joint input and state estimation. Meanwhile, this result gives another way to estimate the state and unknown input simultaneously. Finally, the theoretical result is validated by simulations.

Accurate full‐resolution reconstruction of spike‐encoded image time series using random matrix theory

A new application-specific approach is proposed to reconstruct the full-resolution grey-scale image from its binary spike series. The noisy observation of each pixel value is first computed by counting the spike pulses of a given event duration. The relationship between the light intensity and the signal-dependent noise is utilised in a linear image formation model to compensate truncated intensity error. Then, the minimum variance estimator for the true intensities is derived from this effective model. This estimation problem is non-iteratively solved by applying the asymptotic results of random matrix theory. Experimental results have demonstrated that the algorithm can reliably infer true pixel intensities and is superior to the simple spike counting in terms of both estimation accuracy and robustness.

Estimation of the threshold parameter of a wear-out failure period in the case of a three-parameter Weibull distribution

We aimed to estimate the threshold parameter for the wear-out failure period of a three-parameter Weibull distribution. In this paper, we propose the minimum-variance linear unbiased estimator based on order statistics, which is denoted by 'TBEST'. In Section 2, we verify the validity of TBEST by comparing it with the existing threshold parameter estimator, based on simulation studies of bias and mean squared error (MSE). Our results show that TBEST requires all order statistics, except for the case of an exponential distribution, in which TBEST is reduced to an unbiased estimator based on the smallest observation only. In Section 3, by simulation studies, we compare TBEST and other estimators known to have good performances. In the simulation results, bias and MSE of TBEST were the smallest in most cases. We also show a numerical example to measure fatigue lives in hours of ten bearings from McCool (1974).

The B-exponential divergence and its generalizations with applications to parametric estimation

In this paper a new family of minimum divergence estimators based on the Bregman divergence is proposed, where the defining convex function has an exponential nature. These estimators avoid the necessity of using an intermediate kernel density and many of them also have strong robustness properties. It is further demonstrated that the proposed approach can be extended to construct a class of generalized estimating equations, where the pool of the resultant estimators encompass a large variety of minimum divergence estimators and range from highly robust to fully efficient based on the choice of the tuning parameters. All of the resultant estimators are M-estimators, where the defining functions make explicit use of the form of the parametric model. The properties of these estimators are discussed in detail; the theoretical results are substantiated by simulation and real data examples. It is observed that in many cases, certain robust estimators from the above generalized class provide better compromises between robustness and efficiency compared to the existing standards.

Unbiased Minimum Variance Estimation for Discrete-Time Systems with Measurement Delay and Unknown Measurement Disturbance

This paper addresses the state estimation problem for stochastic systems with unknown measurement disturbances whose any prior information is unknown and measurement delay resulting from the inherent limited bandwidth. For such complex systems, the Kalman-like one-step predictor independent of unknown measurement disturbances is designed based on the linear unbiased minimum variance criterion and the reorganized innovation analysis approach. One simulation example shows the effectiveness of the proposed algorithms.

Principles of Bayesian Inference Using General Divergence Criteria.

When it is acknowledged that all candidate parameterised statistical models are misspecified relative to the data generating process, the decision maker (DM) must currently concern themselves with inference for the parameter value minimising the Kullback–Leibler (KL)-divergence between the model and this process (Walker, 2013). However, it has long been known that minimising the KL-divergence places a large weight on correctly capturing the tails of the sample distribution. As a result, the DM is required to worry about the robustness of their model to tail misspecifications if they want to conduct principled inference. In this paper we alleviate these concerns for the DM. We advance recent methodological developments in general Bayesian updating (Bissiri, Holmes & Walker, 2016) to propose a statistically well principled Bayesian updating of beliefs targeting the minimisation of more general divergence criteria. We improve both the motivation and the statistical foundations of existing Bayesian minimum divergence estimation (Hooker & Vidyashankar, 2014; Ghosh & Basu, 2016), allowing the well principled Bayesian to target predictions from the model that are close to the genuine model in terms of some alternative divergence measure to the KL-divergence. Our principled formulation allows us to consider a broader range of divergences than have previously been considered. In fact, we argue defining the divergence measure forms an important, subjective part of any statistical analysis, and aim to provide some decision theoretic rational for this selection. We illustrate how targeting alternative divergence measures can impact the conclusions of simple inference tasks, and discuss then how our methods might apply to more complicated, high dimensional models.