Classical Hill Estimator Research Articles

Threshold selection and trimming in extremes

We consider removing lower order statistics from the classical Hill estimator in extreme value statistics, and compensating for it by rescaling the remaining terms. Trajectories of these trimmed statistics as a function of the extent of trimming turn out to be quite flat near the optimal threshold value. For the regularly varying case, the classical threshold selection problem in tail estimation is then revisited, both visually via trimmed Hill plots and, for the Hall class, also mathematically via minimizing the expected empirical variance. This leads to a simple threshold selection procedure for the classical Hill estimator which circumvents the estimation of some of the tail characteristics, a problem which is usually the bottleneck in threshold selection. As a by-product, we derive an alternative estimator of the tail index, which assigns more weight to large observations, and works particularly well for relatively lighter tails. A simple ratio statistic routine is suggested to evaluate the goodness of the implied selection of the threshold. We illustrate the favourable performance and the potential of the proposed method with simulation studies and real insurance data.

Priority statement and some properties of t-lgHill estimator

We acknowledge the priority on the introduction of the formula of t-lgHill estimator for the positive extreme value index. We provide a novel motivation for this estimator based on ecologically driven dynamical systems. Another motivation is given directly by applying the general t-Hill procedure to log-gamma distribution. We illustrate the good quality of t-lgHill estimator in comparison to classical Hill estimator on the novel data of the concentration of arsenic in drinking water in the rural area of the Arica and Parinacota Region, Chile.

On the comparison of several classical estimators of the extreme value index

Due to the fact that for heavy tails the classical Hill estimator of a positive extreme value index is asymptotically biased, new and interesting alternative estimators have appeared in the literature. In this work we compare several classical estimators of the extreme value index based on moments of the upper order statistics. Since several alternative estimators have eventually a null asymptotic bias, for some heavy tailed models, the comparison is performed not only with the Hill and recent generalized means estimators but also with an asymptotically unbiased Hill estimator. The comparison study is performed asymptotically, under a third-order framework, and for finite samples, through a Monte Carlo simulation study.

Reduced‐bias kernel estimators of a positive extreme value index

In this paper, we deal with the semi‐parametric estimation of the extreme value index, an important parameter in extreme value analysis. It is well known that many classic estimators, such as the Hill estimator, reveal a strong bias. This problem motivated the study of two classes of kernel estimators. Those classes generalize the classical Hill estimator and have a tuning parameter that enables us to modify the asymptotic mean squared error and eventually to improve their efficiency. Since the improvement in efficiency is not very expressive, we also study new reduced bias estimators based on the two classes of kernel statistics. Under suitable conditions, we prove their asymptotic normality. Moreover, an asymptotic comparison, at optimal levels, shows that the new classes of reduced bias estimators are more efficient than other reduced bias estimator from the literature. An illustration of the finite sample behaviour of the kernel reduced‐bias estimators is also provided through the analysis of a data set in the field of insurance.

On a relationship between randomly and non-randomly thresholded empirical average excesses for heavy tails

Motivated by theoretical similarities between the classical Hill estimator of the tail index of a heavy-tailed distribution and one of its pseudo-estimator versions featuring a non-random threshold, we show a novel asymptotic representation of a class of empirical average excesses above a high random threshold, expressed in terms of order statistics, using their counterparts based on a suitable non-random threshold, which are sums of independent and identically distributed random variables. As a consequence, the analysis of the joint convergence of such empirical average excesses essentially boils down to a combination of Lyapunov’s central limit theorem and the Cramér-Wold device. We illustrate how this allows to improve upon, as well as produce conceptually simpler proofs of, very recent results about the joint convergence of marginal Hill estimators for a random vector with heavy-tailed marginal distributions. These results are then applied to the proof of a convergence result for a tail index estimator when the heavy-tailed variable of interest is randomly right-truncated. New results on the joint convergence of conditional tail moment estimators of a random vector with heavy-tailed marginal distributions are also obtained.

A class of new tail index estimators

In the paper, we propose a new class of functions which is used to construct tail index estimators. Functions from this new class are non-monotone in general, but they are the product of two monotone functions: the power function and the logarithmic function, which play essential role in the classical Hill estimator. The newly introduced generalized moment ratio estimator and generalized Hill estimator have a better asymptotic performance compared with the corresponding classical estimators over the whole range of the parameters that appear in the second-order regular variation condition. Asymptotic normality of the introduced estimators is proved, and comparison (using asymptotic mean square error) with other estimators of the tail index is provided. Some preliminary simulation results are presented.

New Reduced-bias Estimators of a Positive Extreme Value Index

Noting that the classical Hill estimator of a positive extreme value index (EVI) is the logarithm of the mean of order-0 of a set of certain statistics, a more general class of EVI-estimators based on the mean of order-p (MOP), p ⩾ 0, of such statistics was recently introduced. The asymptotic behavior of the class of MOP EVI-estimators is reviewed, and compared to their reduced-bias MOP (RBMOP) and optimal RBMOP versions, which are suggested here and studied both asymptotically and for finite samples, through a large-scale simulation study. Applications to simulated datasets are also put forward.

On an improvement of Hill and some other estimators

In the paper, we propose a new idea in the tail-index estimation. This idea allows us to improve the asymptotic performance of the classical Hill estimator and other most popular estimators over the range of the parameters present in the second-order regular-variation condition. We prove the asymptotic normality of the introduced estimators and provide a comparison (using the asymptotic mean-squared error) with other estimators of the tail index.

A simple generalisation of the Hill estimator

The classical Hill estimator of a positive extreme value index (EVI) can be regarded as the logarithm of the geometric mean, or equivalently the logarithm of the mean of order p=0, of a set of adequate statistics. A simple generalisation of the Hill estimator is now proposed, considering a more general mean of order p≥0 of the same statistics. Apart from the derivation of the asymptotic behaviour of this new class of EVI-estimators, an asymptotic comparison, at optimal levels, of the members of such class and other known EVI-estimators is undertaken. An algorithm for an adaptive estimation of the tuning parameters under play is also provided. A large-scale simulation study and an application to simulated and real data are developed.

Uniform in bandwidth consistency of kernel estimators of the tail index

We consider a class of kernel estimators \(\hat{\tau}_{n,h}\) of the tail index of a Pareto-type distribution, which generalizes and includes the classical Hill estimator \(\hat{a}_{n,k}\). It is well-known that \(\hat{a}_{n,k}\) is a consistent estimator of the tail index if and only if k→ ∞ and k/n→0. Under suitable assumptions on the kernel, \(\hat{\tau} _{n,h}\) is consistent whenever the bandwidth is taken to be a sequence of non-random numbers satisfying hn→0 and nhn→ ∞. We extend this result and prove the consistency uniformly over a certain range of bandwidths. This permits the treatment of estimators of the tail index based upon data-dependent bandwidths, which are often used in practice. In the process, we establish a uniform in bandwidth result for kernel-type regression estimators with a fixed design, which will likely be of separate interest.

Heavy tailed durations of regional rainfall

Durations of rain events and drought events over a given region provide important information about the water resources of the region. Of particular interest is the shape of upper tails of the probability distributions of such durations. Recent research suggests that the underlying probability distributions of such durations have heavy tails of hyperbolic type, across a wide range of spatial scales from 2 km to 120 km. These findings are based on radar measurements of spatially averaged rain rate (SARR) over a tropical oceanic region. The present work performs a nonparametric inference on the Pareto tail-index of wet and dry durations at each of those spatial scales, based on the same data, and compares it with conclusions based on the classical Hill estimator. The results are compared and discussed.

Reduced‐bias tail index estimation and the jackknife methodology

In the context of regularly varying tails, we first analyze a generalization of the classical Hill estimator of a positive tail index, with members that are not asymptotically more efficient than the original one. This has led us to propose alternative classical tail index estimators, that may perform asymptotically better than the Hill estimator. As the improvement is not really significant, we also propose generalized jackknife estimators based on any two members of these two classes. These generalized jackknife estimators are compared with the Hill estimator and other reduced‐bias estimators available in the literature, asymptotically, and for finite samples, through the use of Monte Carlo simulation. The finite‐sample behaviour of the new reduced‐bias estimators is also illustrated through a practical example in the field of finance.

Revisiting the Role of the Jackknife Methodology in the Estimation of a Positive Tail Index

In this article, and in a context of regularly varying tails, we analyze a generalization of the classical Hill estimator of a positive tail index. The members of this general class of estimators are not asymptotically more efficient than the original one, the Hill estimator. We thus propose a class of generalized Jackknife estimators associated with any two members of the first class. They enable the reduction of the main component of bias of the Hill estimator and are dependent of a tuning parameter, which is adequately chosen through an asymptotic variance minimization criterion. These new estimators are compared with the Hill estimator, both asymptotically and for finite samples and, when the underlying distribution is in Hall's class of models, they really improve on the well-known, bias-variance, trade-off characteristic of the Hill estimator.

Semiparametric lower bounds for tail index estimation

We consider estimation of the tail index parameter from i.i.d. observations in Pareto and Weibull type models, using a local and asymptotic approach. The slowly varying function describing the non-tail behavior of the distribution is considered as an infinite dimensional nuisance parameter. Without further regularity conditions, we derive a local asymptotic normality (LAN) result for suitably chosen parametric submodels of the full semiparametric model. From this result, we immediately obtain the optimal rate of convergence of tail index parameter estimators for more specific models previously studied. On top of the optimal rate of convergence, our LAN result also gives the minimal limiting variance of estimators (regular for our parametric model) through the convolution theorem. We show that the classical Hill estimator is regular for the submodels introduced with limiting variance equal to the induced convolution theorem bound. We also discuss the Weibull model in this respect.

Asymptotically best linear unbiased tail estimators under a second-order regular variation condition

For regularly varying tails, estimation of the index of regular variation, or tail index, γ , is often performed through the classical Hill estimator, a statistic strongly dependent on the number k of top order statistics used, and with a high asymptotic bias as k increases, unless the underlying model is a strict Pareto model. First, on the basis of the asymptotic structure of Hill's estimator for different k-values, we propose “asymptotically best linear (BL) unbiased” estimators of the tail index. A similar derivation on the basis of the log-excesses and of the scaled log-spacings is performed and the adequate weights for the largest log-observations are provided. The asymptotic behaviour of those estimators is derived, and they are compared with other alternative estimators, both asymptotically and for finite samples. As an overall conclusion we may say that even asymptotic equivalent estimators may exhibit very diversified finite sample properties.

Estimating catastrophic quantile levels for heavy-tailed distributions

Estimation of the occurrence of extreme events is of prime interest for actuaries. Heavy-tailed distributions are used to model large claims and losses. Within this setting we present a new extreme quantile estimator based on an exponential regression model that was introduced by Feuerverger and Hall [Ann. Stat. 27 (1999) 760] and Beirlant et al. [Extremes 2 (1999) 177]. We also discuss how this approach is to be adjusted in the presence of right censoring. This adaptation can also be linked to robust quantile estimation as this solution is based on a Winsorized mean of extreme order statistics which replaces the classical Hill estimator. We also propose adaptive threshold selection procedures for Weissman’s [J. Am. Stat. Assoc. 73 (1978) 812] quantile estimator which can be used both with and without censoring. Finally some asymptotic results are presented, while small sample properties are compared in a simulation study.

How Can Non-invariant Statistics Work in Our Benefit in the Semi-parametric Estimation of Parameters of Rare Events

In this article, and in a context of regularly varying tails, we suggest a simple generalization of the classical Hill estimator of a positive tail index γ. Such an estimator is merely the Hill estimator associated to artificially shifted data. The shift a imposed to the data is the tuning parameter of the statistical procedure of estimation. Such a tuning parameter enables us, in a great diversity of situations, to reduce the main component of the bias of Hill's estimator, getting thus estimates with stable sample paths around the target value γ, and with small mean squared error.

Maximum likelihood revisited under a semi-parametric context - estimation of the tail index

In this paper, and in a context of regularly varying tails, we study computationally the classical Maximum Likelihood (ML) estimator based on the Paretian behaviour of the excesses over a high threshold, denoted PML-estimator, a type II Censoring estimator based specifically on a Fréchet parent, denoted CENS-estimator, and two ML estimators based on the scaled log-spacings, and denoted SLS-estimators. These estimators are considered under a semi-parametric set-up, and compared with the classical Hill estimator and a Generalized Jackknife (GJ) estimator, which has essentially in mind a reduction of the bias of Hill's estimator.

A class of asymptotically unbiased semi-parametric estimators of the tail index

In this paper we consider a class of consistent semi-parametric estimators of a positive tail index γ, parameterized in atuning orcontrol parameter α. Such a control parameter enables us to have access, for any available sample, to an estimator of γ with a null dominant component of asymptotic bias, and with a reasonably flatMean Squared Error pattern, as a functional ofk, the number of top order statistics considered. Moreover, we are able to achieve a high efficiency relatively to the classical Hill estimator, provided we may have access to a larger number of top order statistics than the number needed for optimal estimation through the Hill estimator.

Comparison of estimators in stable models

In this paper, several estimators of the index and scale parameter of a symmetric univariate stable distribution are analyzed and compared. It turns out that minimum distance estimators with a suitable choice of the weight function have good theoretical and empirical performance. Also some modifications of the moment method are reliable in large parts of the parameter space. The classical Hill estimator, however, works well only for a small stability index and for an extremely large number of observations. One can construct tail-estimators with a considerably improved behaviour compared to the Hill estimator if the scale is known.