Statistical Extreme Value Theory Research Articles

A two-stage monitoring scheme for multiple high-dimensional data streams

With the rapid development of data acquisition and analysis technology, the online monitoring of data streams is particularly important in modern industries. This study proposes an efficient online monitoring scheme for multiple high-dimensional data streams. We first construct a point-wise process monitoring control chart based on statistical extreme value theory. Then we use the clustering method to roughly obtain reliable data streams and suspected abnormal data streams, within the sliding window range, further identify abnormal data streams one by one over time points, and estimate the potential change points of these abnormal streams. Finally, we conduct simulations and case studies on this method, and compare it with existing competitive methods to demonstrate its effectiveness.

Gaidai reliability method for long-term coronavirus modelling

Background Novel coronavirus disease has been recently a concern for worldwide public health. To determine epidemic rate probability at any time in any region of interest, one needs efficient bio-system reliability approach, particularly suitable for multi-regional environmental and health systems, observed over a sufficient period of time, resulting in a reliable long-term forecast of novel coronavirus infection rate. Traditional statistical methods dealing with temporal observations of multi-regional processes do not have the multi-dimensionality advantage, that suggested methodology offers, namely dealing efficiently with multiple regions at the same time and accounting for cross-correlations between different regional observations. Methods Modern multi-dimensional novel statistical method was directly applied to raw clinical data, able to deal with territorial mapping. Novel reliability method based on statistical extreme value theory has been suggested to deal with challenging epidemic forecast. Authors used MATLAB optimization software. Results This paper described a novel bio-system reliability approach, particularly suitable for multi-country environmental and health systems, observed over a sufficient period of time, resulting in a reliable long-term forecast of extreme novel coronavirus death rate probability. Namely, accurate maximum recorded patient numbers are predicted for the years to come for the analyzed provinces. Conclusions The suggested method performed well by supplying not only an estimate but 95% confidence interval as well. Note that suggested methodology is not limited to any specific epidemics or any specific terrain, namely its truly general. The only assumption and limitation is bio-system stationarity, alternatively trend analysis should be performed first. The suggested methodology can be used in various public health applications, based on their clinical survey data.

Statistical modelling of dependence between net demands and deficits in two area power systems

Power system resource adequacy risks is dominated by the extremes of the relevant distributions or processes: the upper tail of demand and the lower tail of available generation. Because of this, relevant data in the historic record are sparse. Moreover, for interconnected systems, the degree of statistical dependence in these extremes between systems can also have a sizeable impact on the level of risk in each system. This paper uses results from statistical extreme value theory (EVT) to fit smoothed joint distributions for the values of (demand minus available renewables) and of surplus/deficit dependence in a two-area system, using data from the Irish and Great Britain power system for examples.As well as the statistical smoothing mitigating the consequences of limited volumes of data, the concept of asymptotic dependence provides a useful explanation of the strength of dependence. There is strong evidence that the deficits in the GB and Ireland systems are asymptotically independent, whereas there is evidence for asymptotic dependence in the distributions of (demand minus available wind capacity). This is consistent with the intuition that, when independent distributions of conventional capacity are convoluted with those of demand and wind, the dependence is weakened. The consequent use of a Gaussian copula to describe the dependence of deficits provides a convenient means of carrying out sensitivity analysis to the strength of relationship.

Gaidai reliability method for long-term coronavirus modelling.

Novel coronavirus disease has been recently a concern for worldwide public health. To determine epidemic rate probability at any time in any region of interest, one needs efficient bio-system reliability approach, particularly suitable for multi-regional environmental and health systems, observed over a sufficient period of time, resulting in a reliable long-term forecast of novel coronavirus infection rate. Traditional statistical methods dealing with temporal observations of multi-regional processes do not have the multi-dimensionality advantage, that suggested methodology offers, namely dealing efficiently with multiple regions at the same time and accounting for cross-correlations between different regional observations. Modern multi-dimensional novel statistical method was directly applied to raw clinical data, able to deal with territorial mapping. Novel reliability method based on statistical extreme value theory has been suggested to deal with challenging epidemic forecast. Authors used MATLAB optimization software. This paper described a novel bio-system reliability approach, particularly suitable for multi-country environmental and health systems, observed over a sufficient period of time, resulting in a reliable long-term forecast of extreme novel coronavirus death rate probability. Namely, accurate maximum recorded patient numbers are predicted for the years to come for the analyzed provinces. The suggested method performed well by supplying not only an estimate but 95% confidence interval as well. Note that suggested methodology is not limited to any specific epidemics or any specific terrain, namely its truly general. The only assumption and limitation is bio-system stationarity, alternatively trend analysis should be performed first. The suggested methodology can be used in various public health applications, based on their clinical survey data.

A matched-filter technique with an objective threshold

We propose an objective threshold determination method for detecting outliers from the empirical distribution of cross-correlation coefficients among seismic waveforms. This method is aimed at detecting seismic signals from continuous waveform records. In our framework, detectability is automatically determined from Akaike’s Information Criterion (AIC). We applied the method of seismic signal detection to continuous records collected over two years. The results show that the maximum value of network cross-correlation coefficients sampled from each constant interval can be approximated by the theory of extreme value statistics, which provides a parametric probability density function of maxima. By using the function, outliers can be considered with a reasonable criterion.

Applying Digital Image Correlation (DIC) Techniques to Characterise Plywood According to UNE-EN 314 Standards: Bonding Quality Tests

The determination of plywood bonding quality by shear testing is one of the most commonly used and informative tests, alongside bending strength, to characterise this compound material. Two types of birch plywood were studied: indoor use (class 1), thickness 18 mm, urea-formaldehyde glue; and exposed outdoor use (class 3), thickness 15 mm, phenolic formaldehyde glue. The corresponding pre-treatments indicated in standards UNE-EN 314-1 and -2 were applied to each type. To test the suitability of the method, the technique of digital image correlation (DIC) was used to obtain information about the complex modes of fracture present in compound materials such as plywood. Assessing the results based on the statistical extreme value theory provided more reliable and robust information about the failure strength values for low failure probabilities. The results confirm that DIC can be used to determine the true distribution of the deformations during the fracture process and, therefore, it provides greater knowledge of the details of the failure process. Statistical evaluation of the experimental data obtained with DIC indicates that test results exceeding the time limit of 30 ± 10 s should be evaluated, contrary to the operating procedure in standard UNE-EN 314-1, which states that they are invalid.

Probabilistic and defect tolerant fatigue assessment of AM materials under size effect

Basic fatigue properties captured by laboratory testing of small specimens are usually applied to safety design of actual components. Nevertheless, their fatigue performances are significantly different due to size effect attributing a fault to possibility of large defects occur. Accordingly, extreme value statistics and weakest-link theory are applicable to defect tolerant fatigue assessment under size effect in this study. Firstly, the basic theory of extreme value statistics and weakest-link are explicated. Then, a probabilistic framework is developed to estimate the fatigue strength/life of Additive manufacturing (AM) material, wherein size effect and fatigue scatter from statistical viewpoint are illustrated and quantitative analysis. In particular, different types of size effect are simplified and unified as defect size variation, and the quantile of the critical defect derived by extreme value statistics is implemented to estimate the bounds of fatigue performance. Finally, three groups of additive manufactured specimens with different volume and processing parameters are utilized for method validation and comparison, which show great capability to estimate the fatigue scatter of AM material under size effect. The goal of this study is also to clarify the potential application of extreme value statistics and weakest-link theory from the probability and defect tolerant perspective.

A novel statistical method for long-term coronavirus modelling

Background: Novel coronavirus disease has been recently a concern for worldwide public health. To determine epidemic rate probability at any time in any region of interest, one needs efficient bio-system reliability approach, particularly suitable for multi-regional environmental and health systems, observed over a sufficient period of time, resulting in a reliable long-term forecast of novel coronavirus infection rate. Traditional statistical methods dealing with temporal observations of multi-regional processes do not have the multi-dimensionality advantage, that suggested methodology offers, namely dealing efficiently with multiple regions at the same time and accounting for cross-correlations between different regional observations. Methods: Modern multi-dimensional novel statistical method was directly applied to raw clinical data, able to deal with territorial mapping. Novel reliability method based on statistical extreme value theory has been suggested to deal with challenging epidemic forecast. Authors used MATLAB optimization software. Results: This paper described a novel bio-system reliability approach, particularly suitable for multi-country environmental and health systems, observed over a sufficient period of time, resulting in a reliable long-term forecast of extreme novel coronavirus death rate probability. Namely, accurate maximum recorded patient numbers are predicted for the years to come for the analyzed provinces. Conclusions: The suggested method performed well by supplying not only an estimate but 95% confidence interval as well. Note that suggested methodology is not limited to any specific epidemics or any specific terrain, namely its truly general. The only assumption and limitation is bio-system stationarity, alternatively trend analysis should be performed first. The suggested methodology can be used in various public health applications, based on their clinical survey data.

Maximum Extreme Flow Estimations in Historical Hydrological Series under the Influence of Decadal Variations

The hypothesis of stationarity is a fundamental condition for the application of the statistical theory of extreme values, especially for climate variables. Decadal-scale fluctuations commonly affect maximum and minimum river discharges. Thus, the probability estimates of extreme events need to be considered to enable the selection of most appropriate time series. The current study proposed a methodology to detect the fluctuation of long wet and dry periods. The study was carried out at the gauging station 4C-001 in Pardo River, State of São Paulo, Brazil. The Spearman, Mann–Kendall and Pettitt’s non-parametric tests were also performed to verify the existence of a temporal trend in the maximum annual daily flows. The graph achieved from the Pettitt’s statistical variable allowed for the identification and separation of the longest dry period (1941 to 1975) and the longest wet period (1976 to 2011), decreasing again in 2012. Analysing the series separately, it was observed that both mean and standard deviation were higher than those corresponding to the dry period. The probable maximum flows for the corrected series showed estimates 10% higher than those estimated for the uncorrected historical series. The proposed methodology provided more realistic estimates for the extreme maximum flows.

Non-regular Frameworks and the Mean-of-Order p Extreme Value Index Estimation

Most of the estimators of parameters of rare and large events, among which we distinguish the extreme value index (EVI) for maxima, one of the primary parameters in statistical extreme value theory, are averages of statistics, based on the k upper observations. They can thus be regarded as the logarithm of the geometric mean, i.e. the logarithm of the power mean of order \(p=0\) of a certain set of statistics. Only for heavy tails, i.e. a positive EVI, quite common in many areas of application, and trying to improve the performance of the classical Hill EVI-estimators, instead of the aforementioned geometric mean, we can more generally consider the power mean of order-p (MO\(_p\)) and build associated MO\(_p\) EVI-estimators. The normal asymptotic behaviour of MO\(_p\) EVI-estimators has already been obtained for \(p<1/(2\xi )\), with consistency achieved for \(p<1/\xi \), where \(\xi \) denotes the EVI. We shall now consider the non-regular case, \(p\ge 1/(2\xi )\), a situation in which either normal or non-normal sum-stable laws can be obtained, together with the possibility of an ‘almost degenerate’ EVI-estimation.

Extreme Low Flow Estimation under Climate Change

Climate change’s impact on water availability has been widely studied, including its impact on very rare values quantified by return levels using the statistical extreme value theory. However, the application of this theory to estimate extreme low flows is barely justified due to a large temporal dependency and a physically highly bounded lower tail. One possible way of overcoming this difficulty is to simulate a very large sample of river flow time series consistent with the observations or the climate projections in order to enable empirical rare percentile estimations. In this paper, such an approach based on simulation is developed and tested for a small mountainous watershed in the French Alps. A bivariate generator of daily temperature and rainfall, developed in collaboration with Paris-Saclay University and based on hidden Markov models, is used to produce a large number of temperature and rainfall time series, further provided as input to a hydrological model to produce a similarly large sample of river flow time series. This sample is statistically analyzed in terms of low flow occurrence and intensity. This framework is adapted to the analysis of both current climate conditions and projected future climate. To study historical low flow situations, the bivariate temperature and rainfall model is fitted to the observed time series while bias-adjusted climate model outputs are used to calibrate the generator for the projections. The approach seems promising and could be further improved for use in more specific studies dedicated to the climate change impact on local low flow situations.

Balanced Open Set Domain Adaptation via Centroid Alignment

Open Set Domain Adaptation (OSDA) is a challenging domain adaptation setting which allows the existence of unknown classes on the target domain. Although existing OSDA methods are good at classifying samples of known classes, they ignore the classification ability for the unknown samples, making them unbalanced OSDA methods. To alleviate this problem, we propose a balanced OSDA methods which could recognize the unknown samples while maintain high classification performance for the known samples. Specifically, to reduce the domain gaps, we first project the features to a hyperspherical latent space. In this space, we propose to bound the centroid deviation angles to not only increase the intra-class compactness but also enlarge the inter-class margins. With the bounded centroid deviation angles, we employ the statistical Extreme Value Theory to recognize the unknown samples that are misclassified into known classes. In addition, to learn better centroids, we propose an improved centroid update strategy based on sample reweighting and adaptive update rate to cooperate with centroid alignment. Experimental results on three OSDA benchmarks verify that our method can significantly outperform the compared methods and reduce the proportion of the unknown samples being misclassified into known classes.

Open-Set Human Identification Based on Gait Radar Micro-Doppler Signatures

Human identification based on gait radar micro-Doppler signatures has received significant attention in recent years. In real world applications, a recognition system should have the ability to reject unknown identities as well as correctly recognize known identities, i.e. the open-set identification. In this paper, we propose a novel framework to tackle this challenging problem. We base our approach on a probabilistic discriminant model built on a deep discriminative representation network (DDRN). Specifically, we first train a DDRN with the cosine margin (CM) loss to map gait samples into an embedding space where the learned features belonging to the same identity are much closer together and those of different identities are further apart. Then, in the learned embedding space, the class-inclusion probability (CIP) model associated with each known identity can be constructed based on the statistical Extreme Value Theory (EVT) to bound each other’s support region, which can then be used to estimate the class-belongingness probabilities of a probe sample. Finally, we threshold on these probabilities to determine whether it belongs to one of the known identities or an unknown class. Experiments on a radar measured gait dataset show the effectiveness of our approach.

Structural change to the persistence of the urban heat island

The term urban heat island (UHI) is used to describe the effect of urban temperatures rising several degrees above concurrent temperatures in surrounding suburban or rural areas. This is typically assessed through records of daily extreme temperatures. However, on a hot day the temperature can exceed an extreme threshold for several consecutive hours, forming a cluster of extremes. We use the statistical theory of extreme values combined with a model that allows structural breaks to show that there has been a significant upward shift in the length of clusters in New York City. No such shift is found at a Connecticut location where the usual UHI assessment indicates that the two sites are comparable. Our study is the first to highlight this danger of the UHI. Prolonged exposure to extreme temperatures has deleterious effects on both health and the environment.

The Use of Extreme Value Theory for Forecasting Long-Term Substation Maximum Electricity Demand

Substation annual maximum electricity demand events are extreme, as customers respond to infrequent and extreme weather. Despite the extreme nature of annual maximum demand, the statistical theory of extreme values has only rarely, if ever, been applied. To support long-term planning, utilities typically complete energy consumption and maximum demand forecasts, which are often conducted separately through two different process, leading to inconsistent trends and messages. To address these shortcomings, a point process approach from extreme value theory is proposed to model substation maximum demand as a function of trends in three common factors already required by utilities including customer count, average demand, and installed photovoltaic capacity. The point process model can be parameterized as a nonstationary generalized extreme value distribution with location and scale parameters dependent on the trends of these factors. As the generalized extreme value distribution governs the behaviors of block maxima (annual maximum demand) with forecast trends of three common factors, substation maximum demand can be estimated as per quantiles required by planning standards. Therefore, the proposed approach is not only realistic and flexible to forecast maximum demand but also ensures consistent outcomes and messaging between the two outputs from energy consumption and maximum demand forecasts.

An Extreme Value Theory Model of Cross-Modal Sensory Information Integration in Modulation of Vertebrate Visual System Functions.

We propose a computational model of vision that describes the integration of cross-modal sensory information between the olfactory and visual systems in zebrafish based on the principles of the statistical extreme value theory. The integration of olfacto-retinal information is mediated by the centrifugal pathway that originates from the olfactory bulb and terminates in the neural retina. Motivation for using extreme value theory stems from physiological evidence suggesting that extremes and not the mean of the cell responses direct cellular activity in the vertebrate brain. We argue that the visual system, as measured by retinal ganglion cell responses in spikes/sec, follows an extreme value process for sensory integration and the increase in visual sensitivity from the olfactory input can be better modeled using extreme value distributions. As zebrafish maintains high evolutionary proximity to mammals, our model can be extended to other vertebrates as well.

Future high-temperature extremes and stationarity

Designing some long-lasting industrial assets necessitates an estimation of far future extremes. Extreme value estimation is commonly based on an application of the statistical extreme value theory (EVT), which requires that the studied variable is independent and identically distributed, or, at least, stationary. Climate variables exhibit different behaviors which potentially violate this assumption: Seasonality is generally the easiest to handle, and interannual variability is more complicated. Now, as far as temperature is concerned, an additional source of non-stationarity appears: the warming trend, whose interactions with interannual variability add another range of complexity. The approach proposed here is based on the construction of a standardized variable, whose extremes can be considered as stationary. This allows an application of EVT in better accordance with its assumptions. Recent works (Parey et al. in J Geophys Res Atmos 118:8285–8296, 2013. https://doi.org/10.1002/jgrd.50629 ) have shown that if optimized smooth trends in mean and standard deviation are removed from the temperature time series, then the extremes of the residuals can be considered as stationary. A statistical test has been designed to check this assumption. Here, the inference of high-temperature extremes from the extremes of this standardized variable and future mean and standard deviation projected at the desired time horizon, and given by climate model simulations, is further analyzed and justified.

Estimating the Maximum Intensities of Soft X-Ray Flares Using Extreme Value Theory

Solar flares are one of the most energetic events in the solar system, their impact on Earth at ground level and its atmosphere remains under study. The repercussions of this phenomenon in our technological infrastructure includes radio blackouts and errors in geopositional and navigation systems that are considered natural hazards in ever more countries. Occurrence frequency and intensity of the most energetic solar flares are been taken into account in national programs for civil protection in order to reduce the risk and increase the resilience from Space Weather events. In this work we use the statistical theory of extreme values as well as other statistical methods in order to asses the magnitudes of the most extreme solar flare events expected to occur in a given period of time. We found that the data set under study presents a dual tail behaviour. Our results show that on average we can expect one solar flare greater than X23 each 25 years, that is to say, one such event each two solar cycles.

Strong disorder leads to scale invariance in complex biological systems

Despite the innate complexity of the cell, emergent scale-invariant behavior is observed in many biological systems. We investigate one example of this phenomenon: the dynamics of large complexes in the bacterial cytoplasm. The observed dynamics of these complexes is scale invariant in three measures of dynamics: mean-squared displacement (MSD), velocity autocorrelation function, and the step-size distribution. To investigate the physical mechanism for this emergent scale invariance, we explore minimal models in which mobility is modeled as diffusion on a rough free-energy landscape in one dimension. We discover that all three scale-invariant characteristics emerge generically in the strong disorder limit. (Strong disorder is defined by the divergence of the ensemble-averaged hop time between lattice sites.) In particular, we demonstrate how the scale invariance of the relative step-size distribution can be understood from the perspective of extreme-value theory in statistics (EVT). We show that the Gumbel scale parameter is simply related to the MSD scaling parameter. The EVT mechanism of scale invariance is expected to be generic to strongly disordered systems and therefore a powerful tool for the analysis of other systems in biology and beyond.

On power-law tail distribution of strength statistics of brittle and quasibrittle structures

Understanding the tail behavior of the probability distribution of structural strength is of paramount importance for reliability-based design of engineering structures. For brittle and quasibrittle structures, the tail distribution not only determines the design strength at a low failure probability, but also governs the functional form of the strength distribution of large-size structures. There exists clear evidence that the Weibull distribution is applicable to brittle structures. Based on the theory of extreme value statistics, the applicability of Weibull distribution suggests that the tail distribution must follow a power law. The justification of the power-law tail distribution was first proposed by Freudenthal based on an assumed type of flaw statistics and linear elastic fracture mechanics of non-interacting flaws. A series of recent studies suggested that the power-law tail can be explained by the transition rate theory governing the statistics of material failure at the nano-scale. In this study, we investigate the tail distribution by considering the randomness in both material strength and applied stress field. The present analysis adopts a nonlocal strength-based failure criterion, in which the spatial variability of the material strength is represented by an autocorrelated random field. The failure statistics of the structure is calculated as a first passage probability. We analyze this problem in 1D, 2D and 3D settings, and the results indicate that in all cases the tail distribution of the nominal structural strength follows a power law. It is shown that the power-law tail behavior of strength distribution stems from the tail distribution of material strength. The flaw statistics introduces additional randomness to the nominal structural strength, but does not dictate the power-law form of its tail distribution.