Two-sample Kolmogorov-Smirnov Statistic Research Articles

Probability density and information entropy of machine learning derived intracranial pressure predictions.

Even with the powerful statistical parameters derived from the Extreme Gradient Boost (XGB) algorithm, it would be advantageous to define the predicted accuracy to the level of a specific case, particularly when the model output is used to guide clinical decision-making. The probability density function (PDF) of the derived intracranial pressure predictions enables the computation of a definite integral around a point estimate, representing the event's probability within a range of values. Seven hold-out test cases used for the external validation of an XGB model underwent retinal vascular pulse and intracranial pressure measurement using modified photoplethysmography and lumbar puncture, respectively. The definite integral ±1 cm water from the median (DIICP) demonstrated a negative and highly significant correlation (-0.5213±0.17, p< 0.004) with the absolute difference between the measured and predicted median intracranial pressure (DiffICPmd). The concordance between the arterial and venous probability density functions was estimated using the two-sample Kolmogorov-Smirnov statistic, extending the distribution agreement across all data points. This parameter showed a statistically significant and positive correlation (0.4942±0.18, p< 0.001) with DiffICPmd. Two cautionary subset cases (Case 8 and Case 9), where disagreement was observed between measured and predicted intracranial pressure, were compared to the seven hold-out test cases. Arterial predictions from both cautionary subset cases converged on a uniform distribution in contrast to all other cases where distributions converged on either log-normal or closely related skewed distributions (gamma, logistic, beta). The mean±standard error of the arterial DIICP from cases 8 and 9 (3.83±0.56%) was lower compared to that of the hold-out test cases (14.14±1.07%) the between group difference was statistically significant (p<0.03). Although the sample size in this analysis was limited, these results support a dual and complementary analysis approach from independently derived retinal arterial and venous non-invasive intracranial pressure predictions. Results suggest that plotting the PDF and calculating the lower order moments, arterial DIICP, and the two sample Kolmogorov-Smirnov statistic may provide individualized predictive accuracy parameters.

Evaluation of Reanalyses over British Columbia. Part I: Daily and Extreme 2-m Temperature

AbstractWeather-station data coverage, quality, and completeness across British Columbia, Canada, degrade outside of population centers and as one goes back in time. This data paucity motivates the search for the best reanalysis to serve as a climatological reference dataset. This study focuses on how well reanalyses represent 2-m temperature (T2M). Systematic error, random error, and two-sample Kolmogorov–Smirnov statistics of daily maximum and minimum T2M are evaluated from the European Centre for Medium-Range Weather Forecasts interim reanalysis (ERA-Interim), the Climate Forecast System Reanalysis (CFSR), the Japanese 55-year Reanalysis (JRA-55), and the Modern-Era Retrospective Analysis for Research and Applications, version 2 (MERRA-2). Also evaluated are the 2- and 30-yr return levels of T2M, which are estimated by the method of L moments from a fitted generalized extreme value (GEV) distribution. Reanalyses are compared with observations from 57 meteorological stations distributed over the complex terrain of British Columbia from 1980 to 2010. Minimum temperatures are better captured than maximum temperatures by all four reanalyses. JRA-55 and ERA-Interim generally perform better across all metrics. Biases are largely explained by poor reanalysis terrain representation. Statistical stationarity over the 30-yr period is assessed by using Gaussian and GEV distributions fitted with and without time-dependent parameters. It is determined that stationary distributions are sufficient to represent the climate of T2M for this region and time period.

Finite sampling inequalities: An application to two-sample Kolmogorov–Smirnov statistics

We review a finite-sampling exponential bound due to Serfling and discuss related exponential bounds for the hypergeometric distribution. We then discuss how such bounds motivate some new results for two-sample empirical processes. Our development complements recent results by Wei and Dudley (2012) concerning exponential bounds for two-sided Kolmogorov–Smirnov statistics by giving corresponding results for one-sided statistics with emphasis on “adjusted” inequalities of the type proved originally by Dvoretzky et al. (1956) [3] and by Massart (1990) for one-sample versions of these statistics.

On the Use of Reanalysis Data for Downscaling

Abstract In this study, a worldwide overview on the expected sensitivity of downscaling studies to reanalysis choice is provided. To this end, the similarity of middle-tropospheric variables—which are important for the development of both dynamical and statistical downscaling schemes—from 40-yr European Centre for Medium-Range Weather Forecasts (ECMWF) Re-Analysis (ERA-40) and NCEP–NCAR reanalysis data on a daily time scale is assessed. For estimating the distributional similarity, two comparable scores are used: the two-sample Kolmogorov–Smirnov statistic and the probability density function (PDF) score. In addition, the similarity of the day-to-day sequences is evaluated with the Pearson correlation coefficient. As the most important results demonstrated, the PDF score is found to be inappropriate if the underlying data follow a mixed distribution. By providing global similarity maps for each variable under study, regions where reanalysis data should not assumed to be “perfect” are detected. In contrast to the geopotential and temperature, significant distributional dissimilarities for specific humidity are found in almost every region of the world. Moreover, for the latter these differences not only occur in the mean, but also in higher-order moments. However, when considering standardized anomalies, distributional and serial dissimilarities are negligible over most extratropical land areas. Since transformed reanalysis data are not appropriate for regional climate models—in opposition to statistical approaches—their results are expected to be more sensitive to reanalysis choice.

Using time‐to‐onset for detecting safety signals in spontaneous reports of adverse events following immunization: a proof of concept study

Disproportionality analyses (DPA) are widely used in pharmacovigilance for detecting safety signals from spontaneous reports of adverse events. In these analyses, time-to-onset (TTO; the time between vaccination and the onset of the adverse event) is rarely considered. Our objective is to assess the potential use of TTO to improve signal detection (SD). Adverse events were defined as signals for a vaccine if the TTO distribution was significantly different from the distribution of other events for the same vaccine and from the distribution obtained with the same event for other vaccines. Distributions were compared within a time window of 30 days by using the two-sample Kolmogorov-Smirnov statistic. With the use of the product label as a proxy of true positive safety signals, TTO SD was compared with a standard DPA method (based on stratified empirical Bayesian geometric mean) for an oral live pediatric vaccine (Rotarix™) and an inactivated adult vaccine (Fluarix™). With the use of the GlaxoSmithKline spontaneous reports database for Rotarix™, 10 Medical Dictionary for Regulatory Activities preferred terms were identified as signals, and among them, five were listed in the product label. The DPA method identified only three preferred terms from the label, that is, TTO SD showed higher sensitivity and specificity. For Fluarix™, TTO SD also showed higher sensitivity but lower specificity. This TTO SD method is complementary, conceptually and practically, to more traditional DPA and does not share the major drawback of DPA known as the masking effect. Higher sensitivity and/or specificity can be achieved using TTO SD.

Testing the difference between two Kolmogorov–Smirnov values in the context of receiver operating characteristic curves

The maximum vertical distance between a receiver operating characteristic (ROC) curve and its chance diagonal is a common measure of effectiveness of the classifier that gives rise to this curve. This measure is known to be equivalent to a two-sample Kolmogorov–Smirnov statistic; so the absolute difference D between two such statistics is often used informally as a measure of difference between the corresponding classifiers. A significance test of D is of great practical interest, but the available Kolmogorov–Smirnov distribution theory precludes easy analytical construction of such a significance test. We, therefore, propose a Monte Carlo procedure for conducting the test, using the binormal model for the underlying ROC curves. We provide Splus/R routines for the computation, tabulate the results for a number of illustrative cases, apply the methods to some practical examples and discuss some implications.

A statistically significant signature of multi-decadal solar activity changes in atmospheric temperatures at three European stations

We study the effect of solar variability on temperatures recorded in three European stations with the longest gapless series available (Prague, Bologna and Uccle). Following a pattern recognition approach, we partition daily temperature “indices” (minimum, maximum and range) in two separate classes with respect to the level of solar activity (high H vs low L 11 year cycles). Using the two-sample Kolmogorov–Smirnov statistics, multiple shuffles of data, and other partitions, we demonstrate that the separation between the probability distribution functions of H and L temperatures is statistically significant and robust. We find that average annual variations for the H and L classes display common and site-specific patterns. For practically all series considered, differences between graphs of annual change for the two classes H and L are large (∼1 °C). Solar activity accounts, at least in part, for the multi-decadal variations in temperature observed at these European sites in the past two centuries.

Randomized rejection procedure for the two-sample Kolmogorov–Smirnov statistic

The two-sample Kolmogorov–Smirnov test is unable to achieve an arbitrary probability of Type I error because it can only take on a limited number of discrete values. We offer a randomized procedure that achieves any specified value of α. We derive formulas for approximating the achievable p-values immediately above and below the desired value of α. For the value of the statistic corresponding to the p-value greater than α, our procedure rejects the null hypothesis randomly with probability sufficient to achieve the specified α. Such a procedure is particularly appropriate for simulation studies. Our procedure leads to a different continuity correction than the one proposed by Kim (J. Amer. Statist. Assoc. 64 (1969) 1625), using the criterion that the continuity correction should cause the randomized procedure to reject the null hypothesis with probability α.

Testing association between species abundance and a continuous variable with Kolmogorov-Smirnov statistics

Association of species abundance with a continuous environmental variable is frequently tested with regression or correlation analyses. However, because these methods ignore the range and frequency distribution of levels of the variable occurring in the study area, they may generate misleading results. We give examples to illustrate the argument. A better approach to test the association between species abundance and a continuous variable should compare levels of the variable in the study area to levels of the variable occurring in sites occupied by the species. If a particular species abundance is not associated with a given continuous variable, then the frequency distribution of levels of this variable measured where individuals of the species occur should mirror the frequency distribution of levels of the variable measured over the study area. We explain how to use the one- and two-sample Kolmogorov-Smirnov statistics to compare the cumulative relative frequencies of levels of the variable where individuals are present with points in the study area. We discuss the statistics, assumptions, limitations, and advantages of these tests.

Identifying different regimes in eruptive activity: An application to Etna volcano

The objective identification of different regimes in the eruptive time-history of a volcano is crucial to the understanding of its physics. While a problem well-known in statistical literature under the name of change-point or scan-point problem, no method of general applicability exists for the identification of different regimes in a time-series. In particular, the available techniques seem unsuitable to the volcanological case. We developed an original procedure based on two-sample Kolmogorov-Smirnov statistics which offers satisfactory accuracy in a broad range of conditions with a minimum of assumptions and is expressly tailored to the study of geophysical phenomena. Our procedure requires neither the a priori knowledge of the number of regimes nor of the statistical distributions governing the whole process, which can be of different type. The parent distribution of each regime is inferred through a goodness-of-fit test, and this in turn allows the confidence intervals for each of the change-points identified to be estimated by numerical simulation. This procedure is applied to the eruptive history of Mount Etna volcano. Available data allow the analysis of flank eruptions in the period 1600–1980, while the total output (summit and flank activity) can be studied only in the period 1971–1981. Information on eruptive history can be therefore obtained at two different timescales. Since no univocally accepted catalog exists except for the last few decades, we use two different sets of data, which practically exhaust all the available information. The results are interpreted by a stability analysis, and only stable results are retained. Our analysis yields that: • - The inter-event times of flank eruptions in the period 1600–1980 follow two regimes before and after year 1865, while the eruptive activity in the period 1971–1981 follows four different regimes. In each regime eruptions occur according to a Poisson process and Etna behaves as a random nonstationary volcano both at long (few centuries) and short (few years) time-scales. The eruptions appear therefore to be triggered by several combining effects with a balance showing fluctuations at both time-scales. • - The volume output of flank eruptions in the period 1600–1980 follows three different regimes, with change-points at 1670 and 1750. The volume output in the period 1971–1981 follows a single regime. Least squares regression and the related confidence bounds, calculated for each regime, suggest that, even accounting for summit activity, the output rate in the period 1971–1981 is not anomalously high. Provided that no further change in regime takes place, the confidence bounds can be used to cast estimates on the expected future activity. • - The series of the mean effusion rates (erupted volume/duration of the eruption) applied to flank eruptions in the period 1600–1980 follows two different regimes, with a change-point around year 1950. There is a net decrease in the mean effusion rate from over 24 m 3 s −1 to about 7 m 3 s −1, which suggests a change in the eruptive style to lower mean effusion rates. The total output (flank + summit) data of the period 1971–1981 confirms this issue, with averaged mean effusion rates lower than 12 m 3 s −1. • - The points of change in regime in the time-series of the inter-eruption times and volume outputs do not coincide, thus implying that eruptions are governed by other factors than the input of magma. At the same time, the analysis of seismic activity does not suggest the stress field as a most important factor. The eruptive activity does not appear to be controlled by a single factor, but by the balance of several contributing factors, further corroborating the conclusion that Etna behaves as a random nonstationary volcano. • - The relatively small fluctuations in each of the periods identified in the flank output series suggest a constant input within each output regime, although the unavailability of both a reliable model and accurate estimates on historical summit eruptions recommends prudence in accepting this conclusion. • - There appears to be no significant correlation between the volume of a “large” flank eruption and the subsequent inter-event time. This, with the above proviso, suggests the absence of any high-level magma reservoir.

Distinct Glycosylation of Serum Proteins in Patients With Cancer: Brief Communication 2

Concanavalin A and wheat germ agglutinin, lectins that interact with serum glycoprotein in a manner similar to the antigen--antibody reaction, were used as "antibodies" in a single radial immunodiffusion technique to test a coded serum panel (from the National Cancer Institute, Bethesda, Md., and the Mayo Clinic, Rochester, Minn.) containing a) 99 serum samples from patients with different types of malignant neoplasms of the gastrointestinal tract, prostate gland, and lung, b) 50 samples from patients with benign diseases of the same organs as those affected in the cancer patients, and c) 50 samples from apparently healthy smokers. The resulting precipitation rings were not correlated to serum protein concentration, and the differences (demonstrated by Student's t-test and with a generalization of the one-sided two-sample Kolmogorov-Smirnov statistic for evaluating diagnostic tests) established that serum glycoproteins are glycosylated differently in cancer patients than in people without cancer.

A Generalization of the One-Sided Two-Sample Kolmogorov-Smirnov Statistic for Evaluating Diagnostic Tests

Suppose a continuous diagnostic measurement is used to classify patients, and suppose E1 false negative errors and E2 false positive errors result. The quantities E1 and E2, and the total number of misclassifications, L = E1 + E2, depend on the choice of cut-off value. We have determined the null distribution of min L, where minimization is over all possible cut-off values. The statistic, min L, can be used as a quick one-sided two-sample test, and min L is also useful for evaluating publications which present only a 2 X 2 table of false positives, false negatives, true positives and true negatives. In such cases, one can use min L to assess the usefulness of the diagnostic measurement, even if one suspects that the authors chose that particular cut-off value which minimized L after looking at the data. We extend these results to a more general weighted loss L = vE1 + MUE2 where v and mu are positive integers, and we show that min L is a generalization of the one-sided two-sample Kolmogorov-Smirnov statistic, and, indeed, exactly equivalent to that statistic for appropriate choices of v and mu.

Critical Values for the One-Sided Two-Sample Kolmogorov-Smirnov Statistic

Abstract We present a new recursion for counting the number of paths which ever attain a given height. Using this recursion, we calculate exact critical values for the one-sided two-sample Kolmogorov-Smirnov statistic for n and m = 3(1)30 and for significance levels α = 0.10, 0.05 and 0.01.

A Simulation Study of One- and Two-Sample Kolmogorov-Smirnov Statistics with a Particular Weight Function

Abstract Critical values for the one-sample and the two-sample Kolmogorov-Smirnov statistics with a particular weight function are compared and found to be greatly different. The difference is shown to be due to the marked sensitivity of the one-sample statistic to the values of the first and last order statistics of the sample distribution.

A Simulation Study of One- and Two-Sample Kolmogorov-Smirnov Statistics with a Particular Weight Function

Abstract Critical values for the one-sample and the two-sample Kolmogorov-Smirnov statistics with a particular weight function are compared and found to be greatly different. The difference is shown to be due to the marked sensitivity of the one-sample statistic to the values of the first and last order statistics of the sample distribution.

On the relative efficiency of some kolmogorov-smirnov-type tests for symmetry about zero

Butler (1969) adapted the two-sample Kolmogorov-Smirnov statistic to obtain a single sample statistic for testing the hypothesis of symmetry about zero, and Chatterjee and Sen (1973) found that the test is more efficient than the sign test. In a manner analogous to Butler's work, the two-sample tests of Kuiper (1960) and Littell (1972), which are often more efficient than the Kolmogorov-Smirnov test, are adapted to test for symmetry.It is found that the adaption of Kuiper's test is always more efficient than Butler's test, and that the adaption of the test of Littell is more efficient than either of the other adaptions in certain cases.

On the Distribution of the Two-Sample Cramer-von Mises Criterion

The Cramer-von Mises $\omega^2$ criterion for testing that a sample, $x_1, \cdots, x_N$, has been drawn from a specified continuous distribution $F(x)$ is \begin{equation*}\tag{1}\omega^2 = \int^\infty_{-\infty} \lbrack F_N(x) - F(x)\rbrack^2 dF(x),\end{equation*} where $F_N(x)$ is the empirical distribution function of the sample; that is, $F_N(x) = k/N$ if exactly $k$ observations are less than or equal to $x(k = 0, 1, \cdots, N)$. If there is a second sample, $y_1, \cdots, y_M$, a test of the hypothesis that the two samples come from the same (unspecified) continuous distribution can be based on the analogue of $N\omega^2$, namely \begin{equation*}\tag{2} T = \lbrack NM/(N + M)\rbrack \int^\infty_{-\infty} \lbrack F_N(x) - G_M(x)\rbrack^2 dH_{N+M}(x),\end{equation*} where $G_M(x)$ is the empirical distribution function of the second sample and $H_{N+M}(x)$ is the empirical distribution function of the two samples together [that is, $(N + M)H_{N+M}(x) = NF_N(x) + MG_M(x)\rbrack$. The limiting distribution of $N\omega^2$ as $N \rightarrow \infty$ has been tabulated [2], and it has been shown ([3], [4a], and [7]) that $T$ has the same limiting distribution as $N \rightarrow \infty, M \rightarrow \infty$, and $N/M \rightarrow \lambda$, where $\lambda$ is any finite positive constant. In this note we consider the distribution of $T$ for small values of $N$ and $M$ and present tables to permit use of the criterion at some conventional significance levels for small values of $N$ and $M$. The limiting distribution seems a surprisingly good approximation to the exact distribution for moderate sample sizes (corresponding to the same feature for $N\omega^2$ [6]). The accuracy of approximation is better than in the case of the two-sample Kolmogorov-Smirnov statistic studied by Hodges [4].