Jackknife Variance Research Articles

Simultaneous computation of Kendall’s tau and its jackknife variance

We present efficient algorithms for simultaneously computing Kendall’s tau and the jackknife estimator of its variance. For the classical pairwise tau, we describe a modification of Knight’s algorithm (originally designed to compute only tau) that does so while preserving its O(nlog2n) runtime in the number of observations n. We also introduce a novel algorithm computing a multivariate extension of tau and its jackknife variance in O(nlog2pn) time.

Heavy alcohol use and the HIV care continuum in Kenya: a population-based study

ABSTRACT Heavy alcohol use (HAU) can destabilize engagement along the HIV care continuum. Population-based studies assessing associations of HAU with HIV treatment outcomes are lacking, especially in sub-Saharan Africa. We leveraged data from the Kenya Population-based HIV Impact Assessment to identify associations of self-reported HAU, assessed using two items measuring the frequency and quantity of past-year alcohol consumption, with serum biomarkers for HIV serostatus unawareness, antiretroviral therapy (ART) non-use, and HIV viremia (≥1000 RNA copies/mL). Overall and sex-stratified survey-weighted logistic regression with jackknife variance estimation modeled adjusted odds ratios (adjOR) of HIV treatment indicators by HAU. Overall, 1491 persons living with HIV aged 15–64 years (68.4% female) were included. The prevalence of HAU was 8.9% (95% confidence interval [95%CI]: 6.8–11.0%) and was significantly more pronounced in males than females (19.6% vs. 4.0%, p < 0.001). In multivariable analysis, HAU was significantly (p < 0.001) associated with HIV serostatus unawareness (adjOR = 3.65, 95%CI: 2.14–6.23), ART non-use (adjOR = 3.81, 95%CI: 2.25–6.43), and HIV viremia (adjOR = 3.13, 95%CI: 1.85–5.32). Incorporating sex-specific alcohol use screening into HIV testing and treatment services in populations where HAU is prevalent could optimize clinical outcomes along the HIV care continuum.

The angular correlation function as measured by the GLEAM-X survey

Abstract The angular correlation is a method for measuring the distribution of structure in the Universe, through the statistical properties of the angular distribution of galaxies on the sky. We measure the angular correlation of galaxies from the second data release of the GaLactic and Extragalactic All-sky Murchison Widefield Array eXtended survey (GLEAM-X) survey, a low-frequency radio survey covering declinations below $+30^\circ$ . We find an angular distribution consistent with the $\Lambda$ CDM cosmological model assuming the best fitting cosmological parameters from Planck Collaboration et al. (2020, A&amp;A, 641, A6). We fit a bias function to the discrete tracers of the underlying matter distribution, finding a bias that evolves with redshift in either a linear or exponential fashion to be a better fit to the data than a constant bias. We perform a covariance analysis to obtain an estimation of the properties of the errors, by analytic, jackknife, and sample variance means. Our results are consistent with previous studies on the topic, and also the predictions of the $\Lambda$ CDM cosmological model.

Leverage, influence, and the jackknife in clustered regression models: Reliable inference using summclust

We introduce a new command, summclust, that summarizes the cluster structure of the dataset for linear regression models with clustered disturbances. The key unit of observation for such a model is the cluster. We therefore propose cluster-level measures of leverage, partial leverage, and influence and show how to compute them quickly in most cases. The measures of leverage and partial leverage can be used as diagnostic tools to identify datasets and regression designs in which cluster–robust inference is likely to be challenging. The measures of influence can provide valuable information about how the results depend on the data in the various clusters. We also show how to calculate two jackknife variance matrix estimators efficiently as a by-product of our other computations. These estimators, which are already available in Stata, are generally more conservative than conventional variance matrix estimators. The summclust command computes all the quantities that we discuss.

Stationary Jackknife

Variance estimation is an important aspect in statistical inference, especially in the dependent data situations. Resampling methods are ideal for solving this problem since these do not require restrictive distributional assumptions. In this paper, we develop a novel resampling method in the Jackknife family called the stationary jackknife. It can be used to estimate the variance of a statistic in the cases where observations are from a general stationary sequence. Unlike the moving block jackknife, the stationary jackknife computes the jackknife replication by deleting a variable length block and the length has a truncated geometric distribution. Under appropriate assumptions, we can show the stationary jackknife variance estimator is a consistent estimator for the case of the sample mean and, more generally, for a class of nonlinear statistics. Further, the stationary jackknife is shown to provide reasonable variance estimation for a wider range of expected block lengths when compared with the moving block jackknife by simulation.

SleepMore

The availability of commercial wearable trackers equipped with features to monitor sleep duration and quality has enabled more useful sleep health monitoring applications and analyses. However, much research has reported the challenge of long-term user retention in sleep monitoring through these modalities. Since modern Internet users own multiple mobile devices, our work explores the possibility of employing ubiquitous mobile devices and passive WiFi sensing techniques to predict sleep duration as the fundamental measure for complementing long-term sleep monitoring initiatives. In this paper, we propose SleepMore, an accurate and easy-to-deploy sleep-tracking approach based on machine learning over the user's WiFi network activity. It first employs a semi-personalized random forest model with an infinitesimal jackknife variance estimation method to classify a user's network activity behavior into sleep and awake states per minute granularity. Through a moving average technique, the system uses these state sequences to estimate the user's nocturnal sleep period and its uncertainty rate. Uncertainty quantification enables SleepMore to overcome the impact of noisy WiFi data that can yield large prediction errors. We validate SleepMore using data from a month-long user study involving 46 college students and draw comparisons with the Oura Ring wearable. Beyond the college campus, we evaluate SleepMore on non-student users of different housing profiles. Our results demonstrate that SleepMore produces statistically indistinguishable sleep statistics from the Oura ring baseline for predictions made within a 5% uncertainty rate. These errors range between 15-28 minutes for determining sleep time and 7-29 minutes for determining wake time, proving statistically significant improvements over prior work. Our in-depth analysis explains the sources of errors.

Interpoint distance-based two-sample tests for functional data

This study examines interpoint distance-based tests for random elements from probability distributions in infinite-dimensional space. Some asymptotic properties such as the limiting distributions and the asymptotic power of the Biswas–Ghosh type test (Biswas and Ghosh in J Multivar Anal 123: 160–171, 2014) are presented using the theory of U-statistic. In addition, the p-value approximation based on the jackknife variance estimators and the Welch–Satterthwaite equation is proposed. Simulation studies are conducted to evaluate the performance of the proposed test statistics in functional data analysis. The proposed tests are shown to have better power than the existing method for functional data in some situations.

A Bootstrap Variance Procedure for the Generalised Regression Estimator

SummaryThe generalised regression estimator (GREG) uses auxiliary data that are available from the finite population to improve the efficiency of the estimator of a total (mean). Estimators of the variance of GREG that have been proposed in the sampling literature include those based on Taylor linearisation and the jackknife techniques. Approximations based on Taylor expansions are reasonable for large samples. However, when the sample size is small, the Taylor‐based variance estimator has a large negative bias. The jackknife variance estimators overestimate the variance of GREG for small sample sizes. We offset these setbacks using a bootstrap procedure for estimating the variance of the GREG. The method uses a bootstrap population constructed with the model underlying the GREG estimator. Repeated samples are selected in the bootstrap population according to the design used to select the initial sample, and the variability associated with these bootstrap samples is used to compute the proposed bootstrap variance estimator. Simulations show that the new bootstrap estimator has a small bias for samples that have few observations.

Inverse weighting method with jackknife variance estimator for differential expression analysis of single-cell RNA sequencing data

Single-cell RNA sequencing (scRNA-seq) data exhibit an unusual abundance of zero counts with a considerable fraction due to the dropout events, which introduces challenges to differential expression analysis. To correct biases in differential expression due to the informative dropouts, an inverse non-dropout-probability weighting method is proposed given that the dropout rate is negatively dependent on the underlying gene expression magnitude in scRNA-seq data. The weights are estimated using the maximum likelihood method where dropout values are integrated out using the Gauss-Hermite quadrature. Linear, generalized linear and mixed regressions with the estimated weights are fitted on original or transformed scRNA-seq data. Variances of coefficient estimators from the weighted regressions are estimated using the jackknife method. Extensive simulation studies are carried out to compare the proposed method to five cutting-edge methods (Limma, edgeR, MAST, ZIAQ and scImpute), where the proposed method performs among the best under all scenarios in terms of AUC, sensitivity, specificity and FDR. Rate of detecting true positives is examined for the proposed method and five comparison methods using mouse embryonic stem cells and fibroblasts where differentially expressed (DE) genes detected in bulk RNA-seq data on the same set of genes under the same conditions from independent source serve as true positives. Specificity is compared for these methods on true negative data by random splitting of a real dataset. Furthermore, the proposed method is illustrated on a lineage study where cells in the same embryo are correlated and genes differentially expressed between cell division lineages are identified.

Weighted U-statistics for likelihood-ratio ordering of bivariate data

Characterisation of marginal distribution and density functions is of interest where data on a pair of random variables (X, Y) are observed. Stochastic orderings between (X, Y) have been studied in statistics and economics. Likelihood-ratio ordering is useful in understanding the behaviour of the random variables. In this article, tests based on U-statistics are proposed to test for equality of marginal density functions against the alternative of likelihood-ratio ordered when (X, Y) are dependent. The tests can be used when the data are either completely observed or subjected to independent univariate right censoring. The asymptotic variances of these tests are complicated and hence, are estimated using jackknife variance estimators. Validity of the jackknife variance estimators in statistical inference based on the proposed tests is demonstrated using simulation studies. The test for uncensored setting has desired size and good power for small sample. The performance of the tests for censored case depends on the sample size, proportion of censoring and the measure of dependence between X and Y. The tests are illustrated on three real data sets chosen in order to bring out various aspects of the tests.

Quantifying uncertainty of subsampling-based ensemble methods under a U-statistic framework

ABSTRACT This paper addresses the problem of variance estimation of predictions obtained from a subsampling-based ensemble estimator, such as subbagging and sub-random forest. We first recognize that a subsampling-based ensemble can be written as an infinite-order U-statistic of degree , where is the subsample size that may depend on the learning sample size n. As a result, one can study the uncertainty of predictions obtained from a subsampling-based ensemble under a U-statistic framework, such as approximating its asymptotic variance. However, existing methods used to estimate the asymptotic variance relies on some regularity conditions. In addition, they tend to yield variance estimations with large bias in finite sample scenarios. Motivated by the work of Wang and Lindsay (2014), we propose to construct an unbiased variance estimator for a subsampling-based ensemble. It is efficient to realize with the help of a partition-resampling scheme. We show by simulation studies that the proposed variance estimator yields better performance in terms of mean, standard deviation, and mean squared error compared to both the infinitesimal jackknife and internal variance estimation methods under either a simple linear regression model or a multivariate adaptive regression splines model. Furthermore, we present how to construct an asymptotic confidence interval for the expected prediction at a given test instance using the proposed variance estimator, and compare its coverage probability to that of competing methods. In the end, we demonstrate the practical applications of the methodology using two real data examples.

Jackknife winsorized variance estimator under imputed data

Abstract In the present study, we consider the problem of missing and extreme values for the estimation of population variance. The presence of extreme values either in the study variable, or the auxiliary variable, or in both of them, can adversely affect the performance of the estimation procedure. We consider three different situations for the presence of extreme values and also consider jackknife variance estimators for the population variance by handling these extreme values under stratified random sampling. Bootstrap technique ABB is carried out to understand the relative relationship more precisely.

Efficient Prediction of Grain Boundary Energies from Atomistic Simulations via Sequential Design

AbstractWith the goal of improving data based materials design, it is shown that by a sequential design of experiment scheme the process of generating and learning from the data can be combined to discover the relevant sections of the parameter space. The application is the energy of grain boundaries as a function of their geometric degrees of freedom, calculated from a simple model, or via atomistic simulations. The challenge is to predict the deep cusps of the energy, which are located at irregular intervals of the geometric parameters. Existing sampling approaches either use large sets of datapoints or a priori knowledge of the cusps' positions. By contrast, the authors' technique can find unknown cusps automatically with a minimal amount of datapoints. Key point is a Kriging interpolator with Matérn kernel to estimate the energy function. Using the jackknife variance, the next point in the sequential design is a compromise between sampling the region of largest fluctuations and avoiding a clustering of datapoints. In this way, the cusps of the energy can be found within only a few iterations, and refined as desired. This approach will be advantageous for any application with strong, localized fluctuations in the values of the unknown function.

Characteristics of users of HIV self-testing in Kenya, outcomes, and factors associated with use: results from a population-based HIV impact assessment, 2018

Background and settingAbout 20% of persons living with HIV aged 15–64 years did not know their HIV status in Kenya, by 2018. Kenya adopted HIV self-testing (HIVST) to help close this gap. We examined the sociodemographic characteristics and outcomes of self-reported users of HIVST as our primary outcome.MethodsWe used data from a 2018 population-based cross-sectional household survey in which we included self-reported sociodemographic and behavioral characteristics and HIV test results. To compare weighted proportions, we used the Rao-Scott χ-square test and Jackknife variance estimation. In addition, we used logistic regression to identify associations of sociodemographic, behavioral, and HIVST utilization.ResultsOf the 23,673 adults who reported having ever tested for HIV, 937 (4.1%) had ever self-tested for HIV. There were regional differences in HIVST, with Nyanza region having the highest prevalence (6.4%), p < 0.001. Factors independently associated with having ever self-tested for HIV were secondary education (adjusted odds ratio [aOR], 3.5 [95% (CI): 2.1–5.9]) compared to no primary education, being in the third (aOR, 1.7 [95% CI: 1.2–2.3]), fourth (aOR, 1.6 [95% CI: 1.1–2.2]), or fifth (aOR, 1.8 [95% CI: 1.2–2.7]) wealth quintiles compared to the poorest quintile and having one lifetime sexual partner (aOR, 1.8 [95% CI: 1.0–3.2]) or having ≥ 2 partners (aOR, 2.1 [95% CI: 1.2–3.7]) compared to none. Participants aged ≥ 50 years had lower odds of self-testing (aOR, 0.6 [95% CI: 0.4–1.0]) than those aged 15–19 years.ConclusionKenya has made progress in rolling out HIVST. However, geographic differences and social demographic factors could influence HIVST use. Therefore, more still needs to be done to scale up the use of HIVST among various subpopulations. Using multiple access models could help ensure equity in access to HIVST. In addition, there is need to determine how HIVST use may influence behavior change towardsaccess to prevention and HIV treatment services.

Estimates of the prevalence of undiagnosed HIV among children living with HIV in Eswatini, Lesotho, Malawi, Namibia, Tanzania, Zambia, and Zimbabwe from 2015 to 2017: an analysis of data from the cross-sectional Population-based HIV Impact Assessment surveys.

In 2020, there were an estimated 1·7 million children younger than 15 years living with HIV worldwide, but there are few data on the proportion of children living with HIV who are undiagnosed. We aimed to estimate the prevalence of undiagnosed HIV among children living with HIV in Eswatini, Lesotho, Malawi, Namibia, Tanzania, Zambia, and Zimbabwe. We conducted an analysis of data from the cross-sectional Population-based HIV Impact Assessment (PHIA) surveys from 2015 to 2017. PHIAs are nationally representative surveys measuring HIV outcomes. HIV rapid test data (with PCR confirmatory testing for children aged <18 months) were used to measure HIV prevalence among children in each country (Eswatini, Lesotho, Malawi, Namibia, Tanzania, Zambia, and Zimbabwe). Mothers or guardians reported previous HIV testing of children and previous results. Detection of antiretroviral medications was done using dried blood spots. Children who tested positive in the PHIA with previous negative or unknown HIV test results and without detectable antiretroviral medication blood concentrations were considered previously undiagnosed; all other children who tested positive were considered previously diagnosed. Survey weights with jackknife variance were used to generate national estimates of HIV prevalence and undiagnosed HIV in children aged 1-14 years. We also report the prevalence (weighted proportions) of antiretroviral therapy coverage and viral load suppression (<400 copies per mL). Between 2015 and 2017, 42 248 children aged 1-14 years were included in the surveys, of whom 594 were living with HIV. Across the seven countries, the estimated weighted HIV prevalence was 0·9% (probability band 0·7-1·1) and we estimated that there were 425 000 (probability band 365 000-485 000) children living with HIV. Among all children living with HIV, 61·0% (n=259 000 [probability band 216 000-303 000]) were previously diagnosed and 39·0% (n=166 000 [128 000-204 000]) had not been previously diagnosed with HIV. Among previously diagnosed children living with HIV, 88·4% had detectable antiretroviral medication blood concentrations and 48·3% had viral load suppression. Among all children living with HIV (regardless of previous diagnosis status), 54·7% had detectable antiretroviral medication blood concentrations and 32·6% had viral load suppression. Our findings show the uneven coverage of paediatric HIV testing across these seven countries and underscore the urgent need to address gaps in diagnosis and treatment for all children living with HIV. None.

Predictors of Palliative Care Knowledge: Findings from the Health Information National Trends Survey.

Palliative care plays an important role in improving the quality of life for patients with cancer and their caregivers and has been associated with increased patient satisfaction. However, palliative care knowledge in the general population is limited, and often physician referral occurs late in prognosis. The objective of this analysis was to examine factors associated with palliative care knowledge. Using data from the 2018 NCI's Health Information National Trends Survey (HINTS) 5 Cycle 2, descriptive statistics, bivariate analyses, and multivariable logistic regressions were used to assess factors associated with respondents' palliative care knowledge using ORs and 95% confidence intervals as measures of association. The outcome of interest was measured with the item "How would you describe your level of knowledge about palliative care?" Possible response selections were "I've never heard of it," "I know a little bit about palliative care," and "I know what palliative care is and could explain it to someone else." To reduce the risk of type 1 error, jackknife variance estimations with repeated replications were used. All analyses were conducted with the SURVEYLOGISTIC command using SAS 9.4 (SAS Institute Inc.), and the statistical significance level was set at P < 0.05. A total of 3,450 respondents (weighted sample size: 249,489,772) met the inclusion criteria. About 89% (n = 3,000) of all respondents had inadequate knowledge of palliative care. Multivariable analyses indicated that frequent health care utilization as defined as ≥ 2 times per year [OR, 3.01; 95% confidence interval (CI), 2.65-3.58], female gender (OR, 2.15; 95% CI, 1.31-3.59), being married (OR, 2.02; 95% CI, 1.14-3.59), having a college degree or higher (OR, 13.83; 95% CI, 1.71-12.04), and having a regular source of care (OR, 2.67; 95% CI, 1.37-1.90) had greater odds of adequate palliative care knowledge. Those without a cancer diagnosis were less likely to have adequate knowledge of palliative care (OR, 0.49; 95% CI, 0.41-0.89). Knowledge of palliative care in the United States is low, particularly for those not already actively using their available healthcare system. Public health education efforts are needed to target subgroups of the U.S. population identified by this analysis to increase palliative care knowledge. Healthcare providers have a major role to play in improving palliative care knowledge.

Estimation of genetic correlation with summary association statistics

SummaryGenome-wide association studies have identified thousands of genetic variants that are associated with complex traits. Many complex traits are shown to share genetic etiology. Although various genetic correlation measures and their estimators have been developed, rigorous statistical analysis of their properties, including their robustness to model assumptions, is still lacking. We develop a method of moments estimator of genetic correlation between two traits in the framework of high-dimensional linear models. We show that the genetic correlation defined based on the regression coefficients and the linkage disequilibrium matrix can be decomposed into both the pleiotropic effects and correlations due to linkage disequilibrium between the causal loci of the two traits. The proposed estimator can be computed from summary association statistics when the raw genotype data are not available. Theoretical properties of the estimator in terms of consistency and asymptotic normality are provided. The proposed estimator is closely related to the estimator from the linkage disequilibrium score regression. However, our analysis reveals that the linkage disequilibrium score regression method does not make full use of the linkage disequilibrium information, and its jackknife variance estimate can be biased when the model assumptions are violated. Simulations and real data analysis results show that the proposed estimator is more robust and has better interpretability than the linkage disequilibrium score regression method under different genetic architectures.

Jackknife empirical likelihood: small bandwidth, sparse network and high-dimensional asymptotics

Summary This article aims to shed light on inference problems for statistical models under alternative or nonstandard asymptotic frameworks from the perspective of the jackknife empirical likelihood. Examples include small-bandwidth asymptotics for semiparametric inference and goodness-of-fit testing, sparse-network asymptotics, many-covariates asymptotics for regression models, and many-weak-instruments asymptotics for instrumental variable regression. We first establish Wilks’ theorem for the jackknife empirical likelihood statistic in a general semiparametric inference problem under the conventional asymptotics. We then show that the jackknife empirical likelihood statistic may lose asymptotic pivotalness in the above nonstandard asymptotic frameworks, and argue that this phenomenon can be understood in terms of the emergence of Efron &amp; Stein (1981)’s bias of the jackknife variance estimator at first order. Finally, we propose a modification of the jackknife empirical likelihood to recover asymptotic pivotalness under both conventional and nonstandard asymptotics. Our modification works for all of the above examples and provides a unified framework for investigating nonstandard asymptotic problems.

An efficient variance estimator of AUC and its applications to binary classification.

The area under the ROC (receiver operating characteristic) curve, AUC, is one of the most commonly used measures to evaluate the performance of a binary classifier. Due to sampling variation, the model with the largest observed AUC score is not necessarily optimal, so it is crucial to assess the variation of AUC estimate. We extend the proposal by Wang and Lindsay and devise an unbiased variance estimator of AUC estimate that is of a two-sample U-statistic form. The proposal can be easily generalized to estimate the variance of a K-sample U-statistic (K ≥ 2). To make our developed variance estimator more applicable, we employ a partition-resampling scheme that is computationally efficient. Simulation studies suggest that the developed AUC variance estimator yields much better or comparable performance to jackknife and bootstrap variance estimators, and computational times that are about 10 to 30 times faster than the times of its counterparts. In practice, the proposal can be used in the one-standard-error rule for model selection, or to construct an asymptotic confidence interval of AUC in binary classification. In addition to conducting simulation studies, we illustrate its practical applications using two real datasets in medical sciences.

Methods for the robust computation of the long-period seismic spectrum of broad-band arrays

SUMMARY We describe array methods to search for low signal-to-noise ratio (SNR) signals in long-period seismic data using Fourier analysis. This is motivated by published results that find evidence of solar free oscillations in the Earth's seismic hum. Previous work used data from only one station. In this paper, we describe methods for computing spectra from array data. Arrays reduce noise level through averaging and provide redundancy that we use to distinguish coherent signal from a random background. We describe two algorithms for calculating a robust spectrum from seismic arrays, an algorithm that automatically removes impulsive transient signals from data, a jackknife method for estimating the variance of the spectrum, and a method for assessing the significance of an entire spectral band. We show examples of their application to data recorded by the Homestake Mine 3-D array in Lead, SD and the Piñon Flats PY array. These are two of the quietest small aperture arrays ever deployed in North America. The underground Homestake data has exceptionally low noise, and the borehole sensors of the PY array also have very low noise, making these arrays well suited to finding very weak signals. We find that our methods remove transient signals effectively from the data so that even low-SNR signals in the seismic background can be found and tested. Additionally, we find that the jackknife variance estimate is comparable to the noise floor, and we present initial evidence for solar g-modes in our data through the T2 test, a multivariate generalization of Student's t-test.