type-I Error Probability Research Articles

A Comparison of Occlusal Schemes with Condylar Inclination and Anterior Guidance in Dentate Individuals.

To compare occlusal schemes with condylar inclination and anterior guidance in dentate individuals. A total of 26 dentate patients of Indian origin between the ages of 18 and 30 with canine-guided occlusion and 26 patients with group function occlusion were included in the study. The School of Articulator Munich (SAM) Axioquick system was used to assess the condylar guidance and eccentric tracings of the patients. For analyses, Student t test was used. For quantitative data, the means and SDs were calculated. For statistical analyses, the probability of type-I error of .05 was considered statistically significant. The mean condylar guidance for canine-guided and group function occlusion was 38.4 ± 12.7 degrees and 30.5 ± 12.5 degrees on the right side, respectively. On the left side it was 36.5 ± 13.0 degrees and 27.5 ± 12.0 degrees for canine-guided and group function occlusion, respectively, with a statistically significant difference (P = .01). The condylar guidance, incisal guidance, Bennett angle, protrusion, left lateral and right lateral movement, and left and right lateral angles were analyzed statistically between the two types of occlusions. The results showed a statistically significant difference between the two groups for all parameters except right condylar guidance. Furthermore, all parameters were higher in the canine-guided occlusion group over group function occlusion. Within the study limitations, it was concluded that the condylar guidance was steeper in canine-guided occlusion than in group function occlusion. The eccentric parameters were steeper in canine-guided occlusion than in group function occlusion. This study showcases that the condylar guidance might not play a major role in determining the occlusal scheme but that the incisal guidance determines the occlusal scheme predominantly in Class 1 occlusion patients.

Operational characteristics of univariate generalized pairwise comparisons with missing data

Generalized pairwise comparisons is a statistical method that allows comparing two groups of subjects based on a set of hierarchically ordered outcomes. It provides a general measure of treatment effect called the net treatment benefit. The method offers a “natural” way of handling missing data: whenever a comparison of two subjects is not possible for a higher priority outcome due to missingness, the comparison is made using the next outcome in the hierarchy. We have investigated the impact of this naïve way of dealing with missing data on the type-I error probability and power of the test. It appears that the naïve net treatment benefit estimator is biased even under missingness completely at random and fails to guarantee the type-I error probability control at the nominal level. Applying inverse probability weighting reduces the bias but does not provide adequate type-I error probability control. Multiple imputation removes the bias and establishes the control of the type-I error probability.

Multi-Hop Network With Multiple Decision Centers Under Expected-Rate Constraints

We consider a multi-hop distributed hypothesis testing problem with multiple decision centers (DCs) for testing against independence and where the observations obey some Markov chain. For this system, we characterize the fundamental type-II error exponents region, i.e., the type-II error exponents that the various DCs can achieve simultaneously, under expected rate-constraints. Our results show that this fundamental exponents region is boosted compared to the region under maximum-rate constraints, and that it depends on the permissible type-I error probabilities. When all DCs have equal permissible type-I error probabilities, the exponents region is rectangular and all DCs can simultaneously achieve their optimal type-II error exponents. When the DCs have different permissible type-I error probabilities, a tradeoff between the type-II error exponents at the different DCs arises. New achievability and converse proofs are presented. For the achievability, a new multiplexing and rate-sharing strategy is proposed. The converse proof is based on applying different change of measure arguments in parallel and on proving asymptotic Markov chains. For the special cases <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">K</i> ∈ {2, 3}, and for arbitrary <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">K</i> ≥ 2 when all permissible type-I error probabilities at the various DCs are equal, we provide simplified expressions for the exponents region; a similar simplification is conjectured for the general case.

INVESTIGATION OF SOME UNIVARIATE NORMALITY TESTS IN TERMS OF TYPE-I ERRORS AND TEST POWER

In this study, Shapiro-Wilk, Kolmogorov-Smirnov, Skewness, Kurtosis, Lilliefors, Jargue-Bera and D'Agostino -Pearson tests, which are univariate normality tests, were compared in point of type-I error and power performances. For comparisons, samples were created in various distributions and sample volumes by simulation technique, and the probability of type-I error was taken as 0.05 in comparisons. Thus, it is aimed to determine the best test to check whether the normality condition is met in univariate data. As a result of the comparison, it was determined that the Jargue-Bera test gave better results than the other tests in point of type-I error probability. In addition, when the normality tests examined in all distributions were taken into account and compared, it was concluded that the Shapiro-Wilk gives better results than other tests in general for normal and non-normal distributions, and that D'Agostino -Pearson, Skewness and Jargue-Bera tests were also stronger than the other tests. In addition, it was determined that the increase in sample sizes increased power of the test. In conclusion, it can be said that in addition to the distribution pattern, type-I error probability and sample size are also very important factors for test power.

Efficient permutation testing of variable importance measures by the example of random forests

Hypothesis testing of variable importance measures (VIMPs) is still the subject of ongoing research. This particularly applies to random forests (RF), for which VIMPs are a popular feature. Among recent developments, heuristic approaches to parametric testing have been proposed whose distributional assumptions are based on empirical evidence. Other formal tests under regularity conditions were derived analytically. But these approaches can be computationally expensive or even practically infeasible. This problem also occurs with non-parametric permutation tests, which are, however, distribution-free and can generically be applied to any kind of prediction model and VIMP. Embracing this advantage, it is proposed to use sequential permutation tests and sequential p-value estimation to reduce the computational costs associated with conventional permutation tests. These costs can be particularly high in case of complex prediction models. Therefore, RF's popular and widely used permutation VIMP (pVIMP) serves as a practical and relevant application example. The results of simulation studies confirm the theoretical properties of the sequential tests, that is, the type-I error probability is controlled at a nominal level and a high power is maintained with considerably fewer permutations needed compared to conventional permutation testing. The numerical stability of the methods is investigated in two additional application studies. In summary, theoretically sound sequential permutation testing of VIMP is possible at greatly reduced computational costs. Recommendations for application are given. A respective implementation for RF's pVIMP is provided through the accompanying R package rfvimptest.

Comparison of the performances of parametric k-sample test procedures as an alternative to one-way analysis of variance

Objectives: The performances of the Welch test, the Alexander-Govern test, the Brown-Forsythe test and the James Second-Order test, which are among the parametric alternatives of one-way analysis of variance and included in the literature, to protect the Type-I error probability determined at the beginning of the trial at a nominal level, were compared with the F test. Methods: Performance of the tests to protect Type-I error; in cases where the variances are homogeneous and heterogeneous, the sample sizes are balanced and unbalanced, the distribution of the data is in accordance with the normal distribution and the log-normal distribution, how it is affected by the change in the number of groups to be compared has been examined on simulation scenarios. Results: The Welch test, the Alexander-Govern test and the James Second-Order test were not affected by the distribution and performed well in situations where variances were heterogeneous. The Brown-Forsythe test was not affected by the distribution, it performed well when the variance was homogeneous and the sample size in the groups to be compared was not equal. Conclusions: The Welch test, the Alexander-Govern test and the James Second-Order test are the tests that can be recommended as an alternative to the F test.

Adaptive Significance Levels in Tests for Linear Regression Models: The e-Value and P-Value Cases

The full Bayesian significance test (FBST) for precise hypotheses is a Bayesian alternative to the traditional significance tests based on p-values. The FBST is characterized by the e-value as an evidence index in favor of the null hypothesis (H). An important practical issue for the implementation of the FBST is to establish how small the evidence against H must be in order to decide for its rejection. In this work, we present a method to find a cutoff value for the e-value in the FBST by minimizing the linear combination of the averaged type-I and type-II error probabilities for a given sample size and also for a given dimensionality of the parameter space. Furthermore, we compare our methodology with the results obtained from the test with adaptive significance level, which presents the capital-P P-value as a decision-making evidence measure. For this purpose, the scenario of linear regression models with unknown variance under the Bayesian approach is considered.

Comparison of the performances of non-parametric k-sample test procedures as an alternative to one-way analysis of variance

Objectives: The performances of the Kruskal-Wallis test, the van der Waerden test, the modified version of Kruskal-Wallis test based on permutation test, the Mood's Median test and the Savage test, which are among the non-parametric alternatives of one-way analysis of variance and included in the literature, to protect the Type-I error probability determined at the beginning of the trial at a nominal level, were compared with the F test. Methods: Performance of the tests to protect Type-I error; in cases where the variances are homogeneous/heterogeneous, the sample sizes are balanced/unbalanced, the distribution of the data is in accordance with the normal distribution/the log-normal distribution, how it is affected by the change in the number of groups to be compared has been examined on simulation scenarios. Results: The Kruskal-Wallis test, the van der Waerden test, the modified version of the Kruskal-Wallis test based on the permutation test were not affected by the distribution of the data, but by the violation of the homogeneity of the variances. The performance of the Mood's Median test and the Savage test were not found to be sufficient in terms of protection of theType-I error compared to other tests. Conclusions: It was determined that the Kruskal-Wallis test, the van der Waerden test, the modified version of Kruskal-Wallis test based on permutation test were not affected by the distribution of the data and tended to preserve the Type-І error when the variances were homogeneous.

On Stein’s lemma in hypotheses testing in general non-asymptotic case

The problem of testing two simple hypotheses in a general probability space is considered. For a fixed type-I error probability, the best exponential decay rate of the type-II error probability is investigated. In regular asymptotic cases (i.e., when the length of the observation interval grows without limit) the best decay rate is given by Stein’s exponent. In the paper, for a general probability space, some non-asymptotic lower and upper bounds for the best rate are derived. These bounds represent pure analytic relations without any limiting operations. In some natural cases, these bounds also give the convergence rate for Stein’s exponent. Some illustrating examples are also provided.

Analysis on Optimal Error Exponents of Binary Classification for Source with Multiple Subclasses

We consider a binary classification problem for a test sequence to determine from which source the sequence is generated. The system classifies the test sequence based on empirically observed (training) sequences obtained from unknown sources and . We analyze the asymptotic fundamental limits of statistical classification for sources with multiple subclasses. We investigate the first- and second-order maximum error exponents under the constraint that the type-I error probability for all pairs of distributions decays exponentially fast and the type-II error probability is upper bounded by a small constant. In this paper, we first give a classifier which achieves the asymptotically maximum error exponent in the class of deterministic classifiers for sources with multiple subclasses, and then provide a characterization of the first-order error exponent. We next provide a characterization of the second-order error exponent in the case where only has multiple subclasses but does not. We generalize our results to classification in the case that and are a stationary and memoryless source and a mixed memoryless source with general mixture, respectively.

Optimizing Type-I (α) and Type-II (β) Error Probabilities by Game-Theoretic Linear Programming for Sequential Sampling Plans in Quality Control

Optimizing Type-I (α) and Type-II (β) Error Probabilities by Game-Theoretic Linear Programming for Sequential Sampling Plans in Quality Control

Operational characteristics of generalized pairwise comparisons for hierarchically ordered endpoints.

The method of generalized pairwise comparisons (GPC) is a multivariate extension of the well-known non-parametric Wilcoxon-Mann-Whitney test. It allows comparing two groups of observations based on multiple hierarchically ordered endpoints, regardless of the number or type of the latter. The summary measure, "net benefit," quantifies the difference between the probabilities that a random observation from one group is doing better than an observation from the opposite group. The method takes into account the correlations between the endpoints. We have performed a simulation study for the case of two hierarchical endpoints to evaluate the impact of their correlation on the type-I error probability and power of the test based on GPC. The simulations show that the power of the GPC test for the primary endpoint is modified if the secondary endpoint is included in the hierarchical GPC analysis. The change in power depends on the correlation between the endpoints. Interestingly, a decrease in power can occur, regardless of whether there is any marginal treatment effect on the secondary endpoint. It appears that the overall power of the hierarchical GPC procedure depends, in a complex manner, on the entire variance-covariance structure of the set of outcomes. Any additional factors (such as thresholds of clinical relevance, drop out, or censoring scheme) will also affect the power and will have to be taken into account when designing a trial based on the hierarchical GPC procedure.

Sequential Classification With Empirically Observed Statistics

Motivated by real-world machine learning applications, we consider a statistical classification task in a sequential setting where test samples arrive sequentially. In addition, the generating distributions are unknown and only a set of empirically sampled sequences are available to a decision maker. The decision maker is tasked to classify a test sequence which is known to be generated according to either one of the distributions. In particular, for the binary case, the decision maker wishes to perform the classification task with minimum number of the test samples, so, at each step, she declares that either hypothesis 1 is true, hypothesis 2 is true, or she requests for an additional test sample. We propose a classifier and analyze the type-I and type-II error probabilities. We demonstrate the significant advantage of our sequential scheme compared to an existing non-sequential classifier proposed by Gutman. Finally, we extend our setup and results to the multi-class classification scenario and again demonstrate that the variable-length nature of the problem affords significant advantages as one can achieve the same set of exponents as Gutman’s fixed-length setting but without having the rejection option.

COVID-19 mortality among kidney transplant candidates is strongly associated with social determinants of health.

The COVID‐19 pandemic has affected all portions of the global population. However, many factors have been shown to be particularly associated with COVID‐19 mortality including demographic characteristics, behavior, comorbidities, and social conditions. Kidney transplant candidates may be particularly vulnerable to COVID‐19 as many are dialysis‐dependent and have comorbid conditions. We examined factors associated with COVID‐19 mortality among kidney transplant candidates from the National Scientific Registry of Transplant Recipients from March 1 to December 1, 2020. We evaluated crude rates and multivariable incident rate ratios (IRR) of COVID‐19 mortality. There were 131 659 candidates during the study period with 3534 all‐cause deaths and 384 denoted a COVID‐19 cause (5.00/1000 person years). Factors associated with increased COVID‐19 mortality included increased age, males, higher body mass index, and diabetes. In addition, Blacks (IRR = 1.96, 95% C.I.: 1.43–2.69) and Hispanics (IRR = 3.38, 95% C.I.: 2.46–4.66) had higher COVID‐19 mortality relative to Whites. Patients with lower educational attainment, high school or less (IRR = 1.93, 95% C.I.: 1.19–3.12, relative to post‐graduate), Medicaid insurance (IRR = 1.73, 95% C.I.: 1.26–2.39, relative to private), residence in most distressed neighborhoods (fifth quintile IRR = 1.93, 95% C.I.: 1.28–2.90, relative to first quintile), and most urban and most rural had higher adjusted rates of COVID‐19 mortality. Among kidney transplant candidates in the United States, social determinants of health in addition to demographic and clinical factors are significantly associated with COVID‐19 mortality.

Asymptotically Optimal One- and Two-Sample Testing With Kernels

We characterize the asymptotic performance of nonparametric one- and two-sample testing. The exponential decay rate or error exponent of the type-II error probability is used as the asymptotic performance metric, and an optimal test achieves the maximum rate subject to a constant level constraint on the type-I error probability. With Sanov's theorem, we derive a sufficient condition for one-sample tests to achieve the optimal error exponent in the universal setting, i.e., for any distribution defining the alternative hypothesis. We then show that two classes of Maximum Mean Discrepancy (MMD) based tests attain the optimal type-II error exponent on \mathbb R <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">d</sup> , while the quadratic-time Kernel Stein Discrepancy (KSD) based tests achieve this optimality with an asymptotic level constraint. For general two-sample testing, however, Sanov's theorem is insufficient to obtain a similar sufficient condition. We proceed to establish an extended version of Sanov's theorem and derive an exact error exponent for the quadratic-time MMD based two-sample tests. The obtained error exponent is further shown to be optimal among all two-sample tests satisfying a given level constraint. Our work hence provides an achievability result for optimal nonparametric one- and two-sample testing in the universal setting. Application to off-line change detection and related issues are also discussed.

Privacy-Utility Tradeoff for Hypothesis Testing Over a Noisy Channel

We study a hypothesis testing problem with a privacy constraint over a noisy channel and derive the performance of optimal tests under the Neyman-Pearson criterion. The fundamental limit of interest is the privacy-utility tradeoff (PUT) between the exponent of the type-II error probability and the leakage of the information source subject to a constant constraint on the type-I error probability. We provide an exact characterization of the asymptotic PUT for any non-vanishing type-I error probability. Our result implies that tolerating a larger type-I error probability cannot improve the PUT. Such a result is known as a strong converse or strong impossibility theorem. To prove the strong converse theorem, we apply the recently proposed technique in (Tyagi and Watanabe, 2020) and further demonstrate its generality. The strong converse theorems for several problems, such as hypothesis testing against independence over a noisy channel (Sreekumar and Gündüz, 2020) and hypothesis testing with communication and privacy constraints (Gilani et al., 2020), are established or recovered as special cases of our result.

Bayesian biomarker-driven outcome-adaptive randomization with an imperfect biomarker assay.

We investigate the impact of biomarker assay's accuracy on the operating characteristics of a Bayesian biomarker-driven outcome-adaptive randomization design. In a simulation study, we assume a trial with two treatments, two biomarker-based strata, and a binary clinical outcome (response). Pbt denotes the probability of response for treatment t (t = 0 or 1) in biomarker stratum (b = 0 or 1). Four different scenarios in terms of true underlying response probabilities are considered: a null (P00 = P01 = 0.25, P10 = P11= 0.25) and consistent (P00 = P10 = 0.25, P01 = 0.5) treatment effect scenario, as well as a quantitative (P00 = P01 = P10 = 0.25, P11 = 0.5) and a qualitative (P00 = P11 = 0.5, P01 = P10 = 0.25) stratum-treatment interaction. For each scenario, we compare the case of a perfect with the case of an imperfect biomarker assay with sensitivity and specificity of 0.8 and 0.7, respectively. In addition, biomarker-positive prevalence values P(B = 1) = 0.2 and 0.5 are investigated. Results show that the use of an imperfect assay affects the operational characteristics of the Bayesian biomarker-based outcome-adaptive randomization design. In particular, the misclassification causes a substantial reduction in power accompanied by a considerable increase in the type-I error probability. The magnitude of these effects depends on the sensitivity and specificity of the assay, as well as on the distribution of the biomarker in the patient population. With an imperfect biomarker assay, the decision to apply a biomarker-based outcome-adaptive randomization design may require careful reflection.

Distributed Hypothesis Testing With Variable-Length Coding

The problem of distributed testing against independence with variable-length coding is considered when the average and not the maximum communication load is constrained as in previous works. The paper characterizes the optimum type-II error exponent of a single-sensor single-decision center system given a maximum type-I error probability when communication is either over a noise-free rate-R link or over a noisy discrete memoryless channel (DMC) with stop-feedback. Specifically, let E denote the maximum allowed type-I error probability. Then the optimum exponent of the system with a rate-R link under a constraint on the average communication load coincides with the optimum exponent of such a system with a rate R/(1-ε) link under a maximum communication load constraint. A strong converse thus does not hold under an average communication load constraint. A similar observation also holds for testing against independence over DMCs. With variable-length coding and stopfeedback and under an average communication load constraint, the optimum type-II error exponent over a DMC of capacity C equals the optimum exponent under fixed-length coding and a maximum communication load constraint when communication is over a DMC of capacity C/(1 - ε).

Assessment of corn silage in dairy farms in the municipality of Patos de Minas, Brazil

This study aimed to assess the corn silage used in 31 dairy farms in themunicipality of Patos de Minas, State of Minas Gerais, Brazil. Aquestionnaire was applied with questions related to the property, the silageprocess, and the use of roughage. A cluster analysis was then performed togroup the properties using the centroid method. Data were submitted toanalysis of variance in a completely randomized design. The continuousvariables and the discrete variables had their means compared using theTukey test and the chi-square test, respectively, with 5% probability of typeI error. Properties with low milk production were those with no technicalassistance; they tended to have nonuniform silo faces and manual removalof silage, resulting in higher silage loss.

On comparing locations of two-parameter exponential distributions using sequential sampling with applications in cancer research

In this paper, we develop testing procedures for hypotheses regarding the difference of location parameters from two independent two-parameter exponential populations with unknown but proportional scale parameters. We design appropriate two-stage and purely sequential procedures to determine the appropriate sample sizes while controlling both type-I and type-II error probabilities at or below the preassigned levels α, β respectively. We prove that both two-stage and purely sequential testing strategies enjoy a number of practically appealing properties. Extensive sets of computer simulations validate our theoretical findings. We illustrate the testing methodologies along with sampling procedures on a cancer studies data.