Infinitesimal Robustness Research Articles

Robust sieve M-estimation with an application to dimensionality reduction

We propose a sieve M-estimation procedure which combines the flexibility of semiparametric inference with the stability and reliability of infinitesimal robustness. We derive the asymptotic theory of the proposed estimators, studying their convergence rate. In the context of functional magnetic resonance imaging (fMRI) data analysis, we illustrate how to apply our procedure to conduct inference on a semiparametric dynamic factor model. Monte Carlo simulations and real data analysis exemplify the stability of our estimators, providing a comparison with the extant, non robust and routinely applied sieve M-estimators.

Privacy-Preserving Parametric Inference: A Case for Robust Statistics

Differential privacy is a cryptographically motivated approach to privacy that has become a very active field of research over the last decade in theoretical computer science and machine learning. In this paradigm, one assumes there is a trusted curator who holds the data of individuals in a database and the goal of privacy is to simultaneously protect individual data while allowing the release of global characteristics of the database. In this setting, we introduce a general framework for parametric inference with differential privacy guarantees. We first obtain differentially private estimators based on bounded influence M-estimators by leveraging their gross-error sensitivity in the calibration of a noise term added to them to ensure privacy. We then show how a similar construction can also be applied to construct differentially private test statistics analogous to the Wald, score, and likelihood ratio tests. We provide statistical guarantees for all our proposals via an asymptotic analysis. An interesting consequence of our results is to further clarify the connection between differential privacy and robust statistics. In particular, we demonstrate that differential privacy is a weaker stability requirement than infinitesimal robustness, and show that robust M-estimators can be easily randomized to guarantee both differential privacy and robustness toward the presence of contaminated data. We illustrate our results both on simulated and real data. Supplementary materials for this article are available online.

M-estimator-based robust Kalman filter for systems with process modeling errors and rank deficient measurement models

The aim of this paper was to develop a robust version of the Kalman filter (KF) to address process modeling errors in linear system with rank deficient measurement models. An important infinitesimal robustness metric, called the influence function, which is widely used in location, scale, and regression models, is introduced into KF. After recasting KF’s update as an artificial linear regression, the influence function of KF is derived in detail. The lack of robustness of KF is clearly demonstrated by the nonboundedness of its influence function. Huber estimator, the most important member of the robust M-estimator family, is introduced into the recast linear regression. The iteration and initialization to iteratively solve the M-estimator are emphasized to address the specific case with process modeling errors and rank deficient measurement models. The iterative reweighted least-squares method together with a covariance construction method is preferred to the standard and simplified Newton’s method and the asymptotic covariance. A new initial value constructed through correcting the a priori estimate to accord with the actual measurement exactly is proposed using the Moore–Penrose pseudoinverse method. A simple but illustrative example is simulated to check the feasibility of the proposed method.

Robust heart rate variability analysis by generalized entropy minimization

Typical heart rate variability (HRV) times series are cluttered with outliers generated by measurement errors, artifacts and ectopic beats. Robust estimation is an important tool in HRV analysis, since it allows clinicians to detect arrhythmia and other anomalous patterns by reducing the impact of outliers. A robust estimator for a flexible class of time series models is proposed and its empirical performance in the context of HRV data analysis is studied. The methodology entails the minimization of a pseudo-likelihood criterion function based on a generalized measure of information. The resulting estimating functions are typically re-descending, which enable reliable detection of anomalous HRV patterns and stable estimates in the presence of outliers. The infinitesimal robustness and the stability properties of the new method are illustrated through numerical simulations and two case studies from the Massachusetts Institute of Technology and Boston’s Beth Israel Hospital data, an important benchmark data set in HRV analysis.

Influence Function Analysis for the Robust Partial Least Squares (RoPLS) Estimator

Partial least squares (PLS) is a class of methods for modeling relations between sets of observed variables by using the latent components where the predictors are highly collinear. SIMPLS is a commonly used PLS algorithm that calculates the latent components directly as linear combinations of the original variables. However, SIMPLS is known to be very sensible to outliers since it is based on the empirical cross-covariance matrix. RoPLS is a recently proposed iterative method for robust SIMPLS. In this article, the influence function for the RoPLS coefficient estimator is derived. It is demonstrated that under certain conditions, the RoPLS estimator has infinitesimal robustness.

Higher-Order Infinitesimal Robustness

Using the von Mises expansion, we study the higher-order infinitesimal robustness of a general M-functional and characterize its second-order properties. We show that second-order robustness is equivalent to the boundedness of both the estimator’s estimating function and its derivative with respect to the parameter. It implies, at the same time, (i) variance robustness and (ii) robustness of higher-order saddlepoint approximations to the estimator’s finite sample density. The proposed construction of second-order robust M-estimators is fairly general and potentially useful in a variety of relevant settings. Besides the theoretical contributions, we discuss the main computational issues and provide an algorithm for the implementation of second-order robust M-estimators. Finally, we illustrate our theory by Monte Carlo simulation and in a real-data estimation of the maximal losses of Nikkei 225 index returns. Our findings indicate that second-order robust estimators can improve on other widely applied robust estimators, in terms of efficiency and robustness, for moderate to small sample sizes and in the presence of deviations from ideal parametric models. Supplementary materials for this article are available online.

Qualitative and infinitesimal robustness of tail-dependent statistical functionals

The main goal of this article is to introduce a new notion of qualitative robustness that applies also to tail-dependent statistical functionals and that allows us to compare statistical functionals in regards to their degree of robustness. By means of new versions of the celebrated Hampel theorem, we show that this degree of robustness can be characterized in terms of certain continuity properties of the statistical functional. The proofs of these results rely on strong uniform Glivenko–Cantelli theorems in fine topologies, which are of independent interest. We also investigate the sensitivity of tail-dependent statistical functionals w.r.t. infinitesimal contaminations, and we introduce a new notion of infinitesimal robustness. The theoretical results are illustrated by means of several examples including general L - and V -functionals.

Robustness and monotonicity properties of generalized correlation coefficients

A new class of generalized correlation coefficients that contains the Pearson and Kendall statistics as special cases was defined by Chinchilli et al. (2005) and applied to the estimation of correlations coefficients within the context of 2×2 cross-over designs for clinical trials. In this paper, we determine the infinitesimal robustness and local stability properties of these generalized correlation coefficients by deriving their corresponding influence functions. For cases in which the population distribution is a bivariate normal or a mixture of bivariate normal distributions we obtain explicit formulas, and establish monotonicity and sign-reverse rule properties of the generalized correlation coefficients.

Optimal robust estimates using the Hellinger distance

Optimal robust M-estimates of a multidimensional parameter are described using Hampel’s infinitesimal approach. The optimal estimates are derived by minimizing a measure of efficiency under the model, subject to a bounded measure of infinitesimal robustness. To this purpose we define measures of efficiency and infinitesimal sensitivity based on the Hellinger distance. We show that these two measures coincide with similar ones defined by Yohai using the Kullback–Leibler divergence, and therefore the corresponding optimal estimates coincide too. We also give an example where we fit a negative binomial distribution to a real dataset of “days of stay in hospital” using the optimal robust estimates.

A very simple robust estimator of a dispersion matrix

In this paper, a new robust estimate of a variance-covariance matrix is defined. This estimate is particularly easy to calculate. Some of its infinitesimal robustness properties are specified. Simulated and real examples are given together with comparisons with classical robust estimates.

The infinitesimal robustness of tests against dependence

The infinitesimal robustness of tests against dependence

Discussion of the Paper by Bruce and Martin

Discussion of the Paper by Bruce and Martin

Some estimation theory on the sphere

We consider distributions with densities of the formf(μ′x) andf(‖x v ‖) where μ andx are unit vectors inR q and ‖x v ‖ is the norm of the part ofx in somes dimensional subspaceV ofR q . For several loss functions, optimal Bayesian and Pitman estimators of μ andV are given. When uniform priors are used, these estimators are identical. Then the infinitesimal robustness characteristics of several special cases of these estimators are calculated.

Most robust M-estimators in the infinitesimal sense

The change-of-variance curve (CVC) is generalized to M-estimators with piecewise continuous ψ-functions, in which case it becomes a Schwartz distribution. An M-estimator is called most B-robust when it minimizes Hampel's gross-error sensitivity γ *, and most V-robust when it minimizes the change-of-variance sensitivity k *. In the general case, the median is most B-robust and most V-robust. If consideration is restricted to redescending M-estimators, then the skipped median is most B-robust and the median-type tanh-estimator is most V-robust. By means of these results, complete solutions of the problems of optimal infinitesimal robustness are obtained.

Influence curves of general statistics

The influence curve (JC) of a Fisher-consistent functional was introduced by F. Hampel and plays a central role in the search for robust estimators. An extension of this notion to non-Fisher-consistent functionals is proposed in order to investigate the infinitesimal robustness of more general statistics, e.g. those used in hypothesis testing. This new definition inherits many useful properties, including some on asymptotic efficiency. Functionals in two variables, arising from two-sample statistics, are treated too. Connections with Hodges-Lehmann shift estimators are discovered. One- and two-sample rank statistics illustrate the theory.

A new infinitesimal approach to robust estimation

The infinitesimal robustness of the asymptotic variance of location M-estimators is investigated by means of the change-of-variance curve (CVC), which bears some resemblance to the influence curve (IC). It is proved that this CVC leads to a more stringent robustness property than the IC and that the Huber estimators are still optimal in this new sense.