Statistical Testing Procedure Research Articles

Statistical fractal analysis in testing the Hypotheses with imprecise data

A new statistical hypothesis testing procedure using fractal analysis, incorporating the concept of lacunarity for interval population parameters when the sample data are real intervals, is introduced. The decision rules for accepting or rejecting the null and alternative hypotheses are provided, along with testing procedures and numerical examples. Additionally, the proposed tests are extended to statistical hypotheses involving fuzzy data samples with lacunarity.

A Dunnett-Type Test and Its Sample Size Calculation for Comparing K ROC Curves with a Control.

Diagnostic biomarkers are key components of diagnostics. In this paper, we consider diagnostic biomarkers taking continuous values that are associated with a dichotomous disease status, called malignant or benign. The performance of such a biomarker is evaluated by the area under the curve (AUC) of its receiver operating characteristic curve. We assume that, together with the disease status, one control and multiple experimental biomarkers are collected from each subject to test if any of the experimental biomarkers have a larger AUC than the control. In this case, each experimental biomarker will be compared with the control so that a multiple testing issue is involved in the comparisons. In this paper, we propose a simple non-parametric statistical testing procedure to compare K(≥2) experimental biomarkers with a control, adjusting for the multiplicity and its sample size calculation method. Our sample size formula requires the specification of the AUC values (or the standardized effect size of each biomarker between the benign and malignant groups) together with the correlation coefficients between the biomarkers, the prevalence of the malignant group in the study population, the type I error rate, and the power. Through simulations, we show that the statistical test controls the overall type I error rate accurately and the proposed sample size closely maintains the specified statistical power.

Multiple testing of interval composite null hypotheses using randomized p-values

Equivalence tests are statistical hypothesis testing procedures that aim to establish practical equivalence rather than the usual statistical significant difference. These testing procedures are frequent in “bioequivalence studies," where one would wish to show that, for example, an existing drug and a new one under development have comparable therapeutic effects. In this article, we propose a two-stage randomized (RAND2) p-value that depends on a uniformly most powerful (UMP) p-value and an arbitrary tuning parameter c∈[0,1]\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$c\\in [0,1]$$\\end{document} for testing an interval composite null hypothesis. We investigate the behavior of the distribution function of the two p-values under the null hypothesis and alternative hypothesis for a fixed significance level t∈(0,1)\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$t\\in (0,1)$$\\end{document} and varying sample sizes. We evaluate the performance of the two p-values in estimating the proportion of true null hypotheses in multiple testing. We conduct a family-wise error rate control using an adaptive Bonferroni procedure with a plug-in estimator to account for the multiplicity that arises from our multiple hypotheses under consideration. The various claims in this research are verified using a simulation study and real-world data analysis.

Inference for Dependent Data with Learned Clusters

Abstract This paper presents and analyzes an approach to cluster-based inference for dependent data. The primary setting considered here is with spatially indexed data in which the dependence structure of observed random variables is characterized by a known, observed dissimilarity measure over spatial indices. Observations are partitioned into clusters with the use of an unsupervised clustering algorithm applied to the dissimilarity measure. Once the partition into clusters is learned, a cluster-based inference procedure is applied to a statistical hypothesis testing procedure. The procedure proposed in the paper allows the number of clusters to depend on the data, which gives researchers a principled method for choosing an appropriate clustering level. The paper gives conditions under which the proposed procedure asymptotically attains correct size. A simulation study shows that the proposed procedure attains near nominal size in finite samples in a variety of statistical testing problems with dependent data.

A model-based hierarchical Bayesian approach to Sholl analysis.

Due to the link between microglial morphology and function, morphological changes in microglia are frequently used to identify pathological immune responses in the central nervous system. In the absence of pathology, microglia are responsible for maintaining homeostasis, and their morphology can be indicative of how the healthy brain behaves in the presence of external stimuli and genetic differences. Despite recent interest in high throughput methods for morphological analysis, Sholl analysis is still widely used for quantifying microglia morphology via imaging data. Often, the raw data are naturally hierarchical, minimally including many cells per image and many images per animal. However, existing methods for performing downstream inference on Sholl data rely on truncating this hierarchy so rudimentary statistical testing procedures can be used. To fill this longstanding gap, we introduce a parametric hierarchical Bayesian model-based approach for analyzing Sholl data, so that inference can be performed without aggressive reduction of otherwise very rich data. We apply our model to real data and perform simulation studies comparing the proposed method with a popular alternative. Software to reproduce the results presented in this article is available at: https://github.com/vonkaenelerik/hierarchical_sholl. An R package implementing the proposed models is available at: https://github.com/vonkaenelerik/ShollBayes.

Network Activity Evaluation Reveals Significant Gene Regulatory Architectures during SARS-CoV-2 Viral Infection from Dynamic scRNA-seq Data.

The key to understand COVID-19 caused by SARS-CoV-2, which has caused massive deaths worldwide, is to reveal the gene activities at molecular level. Single-cell RNA-sequencing (scRNA-seq) technology allows us to capture gene expression at high resolution, thereby delineating cell-specific gene regulatory network (GRN). Network activity refers to the degree of consistency between GRN architectures and gene expression profiles in a specific condition or cellular microenvironment. Currently, numerous experimentally determined molecular interactions, including regulatory relationships closely related to SARS-CoV-2 infection, are documented in knowledge-bases. However, GRN activity is closely related to the cell dynamic environment and the heterogeneity of cell clusters. Therefore, to evaluate the consistency of GRN with gene expression profiles, we propose a single-cell Network Activity Evaluation framework, called scNAE. First, scNAE performs ODE modeling of time-course gene expression data. Then, the loss function with regularization penalty terms is constructed for formulating GRN inference rules from transcriptomic data. Furthermore, we have devised a rapid-convergence alternating direction method of multipliers to solve the regularized and constrained programs. Finally, an empirical P-value is derived based on a permutation statistical testing procedure to quantify the likelihood significance of the network matching with the data. The efficiency and advantage of scNAE have also been demonstrated by extensive numerical experiments, which can clearly depict the dynamic responses underlying GRN architectures triggered by the infection of SARS-CoV-2 in cells. The code and data of scNAE are available at https://github.com/zpliulab/scNAE.

The reliability of parametric methods in the case of rating scales: A simulation study

AbstractA recurring question is whether rating scales should be considered metrically scaled or merely ordinally scaled. This has direct implications for the permissible statistical procedures for significance testing. Based on the results of a simulation study, it is shown that the use of parametric procedures for rating scales has distinct advantages over the nonparametric alternatives. It is also shown that the parametric procedures are robust to violations of the assumption of normality, which only result in a modest loss of power compared with continuous variables. This loss should be taken into account when calculating the optimal sample size. The results suggest that sample sizes about 25% larger should be chosen for discrete rating scales than for continuous variables.

Granger causality tests based on reduced variable information

Granger causality is a classical and important technique for measuring predictability from one group of time series to another by incorporating information of the variables described by a full vector autoregressive (VAR) process. However, in some applications economic forecasts need to be made based on information provided merely by a portion of variates (e.g., removal of a listed stock due to halting, suspension or delisting). This requires a new formulation of forecast based on an embedded subprocess of VAR, whose theoretical properties are often difficult to obtain. To avoid the issue of identifying the VAR subprocess, we propose a computation‐based approach so that sophisticated predictions can be made by utilizing a reduced variable information set estimated from sampled data. Such estimated information set allows us to develop a suitable statistical hypothesis testing procedure for characterizing all designated Granger causal relationships, as well as a useful graphical tool for presenting the causal structure over the prediction horizon. Finally, simulated data and a real example from the stock markets are used to illustrate the proposed method.

Visual Data Exploration as a Statistical Testing Procedure: Within-View and Between-View Multiple Comparisons.

A fundamental problem in visual data exploration concerns whether observed patterns are true or merely random noise. This problem is especially pertinent in visual analytics, where the user is presented with a barrage of patterns, without any guarantees of their statistical validity. Recently this problem has been formulated in terms of statistical testing and the multiple comparisons problem. In this paper, we identify two levels of multiple comparisons problems in visualization: the within-view and the between-view problem. We develop a statistical testing procedure for interactive data exploration that controls the family-wise error rate on both levels. The procedure enables the user to determine the compatibility of their assumptions about the data with visually observed patterns. We present use-cases where we visualize and evaluate patterns in real-world data.

Statistical test for detrending-moving-average-based multivariate regression model

The detrending-moving-average-based multivariate linear regression has been recently introduced as a new tool to probe into multiscale dependent behavior between multivariate data. It well solves the dependence among the nonstationary variates while classic multivariate linear regression dose not. The real-world applications demonstrate its versatility and robustness. However, the statistical significance of this new model needs to be further improved mathematically not just empirically. Hence, the main motivation of this paper is to provide a solid theoretical basis for the latest regression framework. Specifically, four propositions and three theorems are deduced theoretically to support the statistical significance test of detrending-moving-average-based multivariate linear regression model. These propositions and theorems revolve around hypothesis testing of each partial regression coefficients and regression model. They can examine the statistical significance at every given scales. Moreover, we also resort to synthetic time series to verify the performances of the proposed statistical test procedure. By using those statistical tests, we investigate the multiscale interdependence between the crude oil price and the exchange rates among oil-importing and oil-exporting countries. Some interesting findings show that the dependent behavior between the oil price and exchange rates varies at various scales, which cannot be achieved from classical multivariate regression model.

Applying Joint Graph Embedding to Study Alzheimer's Neurodegeneration Patterns in Volumetric Data.

Neurodegeneration measured through volumetry in MRI is recognized as a potential Alzheimer's Disease (AD) biomarker, but its utility is limited by lack of specificity. Quantifying spatial patterns of neurodegeneration on a whole brain scale rather than locally may help address this. In this work, we turn to network based analyses and extend a graph embedding algorithm to study morphometric connectivity from volume-change correlations measured with structural MRI on the timescale of years. We model our data with the multiple random eigengraphs framework, as well as modify and implement a multigraph embedding algorithm proposed earlier to estimate a low dimensional embedding of the networks. Our version of the algorithm guarantees meaningful finite-sample results and estimates maximum likelihood edge probabilities from population-specific network modes and subject-specific loadings. Furthermore, we propose and implement a novel statistical testing procedure to analyze group differences after accounting for confounders and locate significant structures during AD neurodegeneration. Family-wise error rate is controlled at 5% using permutation testing on the maximum statistic. We show that results from our analysis reveal networks dominated by known structures associated to AD neurodegeneration, indicating the framework has promise for studying AD. Furthermore, we find network-structure tuples that are not found with traditional methods in the field.

Detection-only versus detection and identification of model misspecifications

It is common practice to use the well-known concept of the minimal detectable bias (MDB) to assess the performance of statistical testing procedures. However, such procedures are usually applied to a null and a set of multiple alternative hypotheses with the aim of selecting the most likely one. Therefore, in the DIA method for the detection, identification and adaptation of model misspecifications, rejection of the null hypothesis is followed by identification of the potential source of the model misspecification. With identification included, the MDBs do not truly reflect the capability of the testing procedure and should therefore be replaced by the minimal identifiable bias (MIB). In this contribution, we analyse the MDB and the MIB, highlight their differences, and describe their impact on the nonlinear DIA-estimator of the model parameters. As the DIA-estimator inherits all the probabilistic properties of the testing procedure, the differences in the MDB and MIB propagation will also reveal the different consequences a detection-only approach has versus a detection+identification approach. Numerical algorithms are presented for computing the MDB and the MIB and also their effect on the DIA-estimator. These algorithms are then applied to a number of examples so as to analyse and illustrate the different concepts.

Statistical Inferences for Complex Dependence of Multimodal Imaging Data

Statistical analysis of multimodal imaging data is a challenging task, since the data involves high-dimensionality, strong spatial correlations and complex data structures. In this article, we propose rigorous statistical testing procedures for making inferences on the complex dependence of multimodal imaging data. Motivated by the analysis of multi-task fMRI data in the Human Connectome Project (HCP) study, we particularly address three hypothesis testing problems: (a) testing independence among imaging modalities over brain regions, (b) testing independence between brain regions within imaging modalities, and (c) testing independence between brain regions across different modalities. Considering a general form for all the three tests, we develop a global testing procedure and a multiple testing procedure controlling the false discovery rate. We study theoretical properties of the proposed tests and develop a computationally efficient distributed algorithm. The proposed methods and theory are general and relevant for many statistical problems of testing independence structure among the components of high-dimensional random vectors with arbitrary dependence structures. We also illustrate our proposed methods via extensive simulations and analysis of five task fMRI contrast maps in the HCP study. Supplementary materials for this article are available online.

Machine learning models for predicting steroid-resistant of nephrotic syndrome.

In the absence of effective measures to predict steroid responsiveness, patients with nonhereditary steroid-resistant nephrotic syndrome (SRNS) have a significantly increased risk of progression to end-stage renal disease. In view of the poor outcomes of SRNS, it is urgent to identify the steroid responsiveness of idiopathic nephrotic syndrome (INS) early. To build a prediction model for SRNS, we collected 91 subjects; 57 of them had steroid-sensitive nephrotic syndrome, and the others had SRNS. For each subject, 87 clinical variables were measured. In general, only a small part of these variables is informative to SRNS. Thus, we proposed a new variable selection framework including a penalized regression approach (named MLR+TLP) to select variables having a linear effect on the SRNS and a nonparametric screening method (MAC) to select variables having a nonlinear marginal (joint) effect on the SRNS. Thereafter, considering the correlation between selected clinical variables, we used a stepwise method to build our final model for predicting SRNS. In addition, a statistical testing procedure is proposed to test the overfitting of the proposed model. Twenty-six clinical variables were selected to be informative to SRNS, and an SVM model was built to predict SRNS with a leave-one-out cross-validation (LOO-CV) accuracy of 95.2% (overfitting p value<0.005). To make the model more useful, we incorporate prior medical information into the model and consider the correlation between selected variables. Then, a reduced SVM model including only eight clinical variables (erythrocyte sedimentation rate, urine occult blood, percentage of neutrophils, immunoglobulin A, cholesterol, vinculin autoantibody, aspartate aminotransferase, and prolonged prothrombin time) was built to have a LOO-CV accuracy of 92.8% (overfitting p value<0.005). The validation cohort showed that the reduced model obtained an accuracy of 94.0% (overfitting p value<0.005), with a sensitivity of 90.0% and a specificity of 96.7%. Notably, vinculin autoantibody is the only podocyte autoantibody included in this model. It is linearly related to steroid responsiveness. Finally, our model is freely available as a user-friendly web tool at https://datalinkx.shinyapps.io/srns/. The SRNS prediction model constructed in this study comprehensively and objectively evaluates the internal conditions and disease status of INS patients and will provide scientific guidance for selecting treatment methods for children with nonhereditary SRNS.

A Flexible Zero-Inflated Poisson-Gamma Model with Application to Microbiome Sequence Count Data

In microbiome studies, it is of interest to use a sample from a population of microbes, such as the gut microbiota community, to estimate the population proportion of these taxa. However, due to biases introduced in sampling and preprocessing steps, these observed taxa abundances may not reflect true taxa abundance patterns in the ecosystem. Repeated measures, including longitudinal study designs, may be potential solutions to mitigate the discrepancy between observed abundances and true underlying abundances. Yet, widely observed zero-inflation and over-dispersion issues can distort downstream statistical analyses aiming to associate taxa abundances with covariates of interest. To this end, we propose a Zero-Inflated Poisson Gamma (ZIPG) model framework to address these aforementioned challenges. From a perspective of measurement errors, we accommodate the discrepancy between observations and truths by decomposing the mean parameter in Poisson regression into a true abundance level and a multiplicative measurement of sampling variability from the microbial ecosystem. Then, we provide a flexible ZIPG model framework by connecting both the mean abundance and the variability of abundances to different covariates, and build valid statistical inference procedures for both parameter estimation and hypothesis testing. Through comprehensive simulation studies and real data applications, the proposed ZIPG method provides significant insights into distinguished differential variability and mean abundance. Supplementary materials for this article are available online.

Evaluating Treatment Efficacy by Combining Multiple Measures in Clinical Trial Applications.

A variety of clinical and laboratory measures can be used in clinical trials to assess the benefit of a new treatment over the standard of care. Data from clinical studies are often analyzed by combining individual outcomes into one primary outcome. That primary outcome is then referred to as a composite endpoint or a combined endpoint. We propose an analysis on the composite endpoint with Gehan's (1965) ranking approach where each subject in the treatment group is compared with each subject in the control group in a pair-wise manner. Our approach reduces computational time and complexity to construct a subject-level pairwise composite score. We develop a statistical testing procedure for the analysis of composite endpoints when using the hierarchical scores. In this article, we propose two tests (a parametric test and a non-parametric bootstrap procedure) for evaluating the effect of treatment. The proposed parametric test has an asymptotic F-distribution based on standard statistical assumptions. We conduct an extensive simulation study to assess the operating characteristics of the proposed methods and to compare them with an existing method. We illustrate the methods using publicly available data from two clinical studies.

Statistical Procedures for State Degradation Testing by Vibration Methods

Abstract This paper presents selected problems of the statistical evaluation of machines with the use of vibration diagnostics methods. The state of knowledge of this issue indicates the need to update the description and improve the issues related to the acquisition and processing of data, the analysis of the effectiveness of diagnosis procedures, optimization of the set and selection of diagnostic parameters, with statistical support. New proposals of statistical software solutions and various aspects of theoretically justified partial procedures indicate important problems for a good diagnosis.

Chicken eggshells as a soil amendment and their relationship with the morphological response of mustard plants (Brassica juncea, L.)

The purpose of this study is to obtain objective information about the dosage of the use of chicken egg shells as a soil reformer in the sustainable cultivation of mustard plants. The research has been carried out at the Coronary Garden, Faculty of Agriculture, Syiah Kuala University from August to September 2021. The study used polybags in the form of experiments with a completely randomized design. The treatment tried was in the form of various doses of chicken eggshell flour (0, 5, 10, 15, and 20 g / stem), and the morphological response of mustard plants were measured in the form of; plant height; the number of leaves per plant; fresh weight; and the dry weight of the plant. To test the effect of treatment on the response variables of mustard plants using the ANOVA statistical test procedure, while to test the difference in the average value of plant responses using the Tukey HSD test at α 0.05. Mustard plants showed a positive morphological response to the administration of chicken shell flour as seen from plant height, the number of leaves and wet weight, and dry weight. Chicken eggshell flour as a soil enhancer contains macro and micro nutrients to increase the growth and yield of mustard rice fields. The use of chicken eggshell flour of 5 g / stem provides the best morphological response for mustard plants.

Neuroscience Needs to Test Both Statistical and Scientific Hypotheses.

Experimental neuroscience typically uses "p-valued" statistical testing procedures (null hypothesis significance testing; NHST) in evaluating its results. The rote, often misguided, application of NHST (Gigerenzer, 2008) has led to errors and "questionable research practices." Although the problems could be avoided with better statistics training (Lakens, 2021), there have been calls to abandon NHST altogether. One suggestion is to replace NHST with "estimation statistics" (Cumming and Calin-Jageman, 2017; Calin-Jageman and Cumming, 2019). Estimation statistics emphasizes the uncertainty inherent in scientific investigations and uses metrics, e.g., confidence intervals (CIs), that draw attention to uncertainty. Besides procedural steps and methods, the Estimation Approach prefers expressing "quantitative," rather than "qualitative" conclusions and making generalizations, rather than testing scientific hypotheses. The Estimation Approach embodies a philosophy of science-its ultimate goals, experimental mindset, and specific aims-that diverges unhelpfully from what laboratory-based neuroscience needs. The Estimation Approach meshes naturally with, e.g., clinical neuroscience, drug development, human psychology, and social sciences. It fits less well with much of the neuroscience published in the Journal of Neuroscience, for example. In contrast, the philosophy behind NHST fits naturally with traditional, evaluative testing of scientific hypotheses. Finally, some Estimation Approach remedies, e.g., replication, ideally with "preregistration," are incompatible with much experimental neuroscience. This Dual Perspective essay argues that, while neuroscience can benefit from practical aspects of estimation statistics, entirely replacing conventional methods with the Estimation Approach would be a mistake. NHST testing should be retained and improved.SIGNIFICANCE STATEMENT Experimental neuroscience relies on statistical procedures to assess the meaning and importance of its research findings. Optimal scientific communication demands a common set of assumptions for expressing and evaluating results. Problems arising from misuse of conventional significance testing methods have led to a proposal to replace significance testing with an Estimation Statistics Approach. Practical elements of the Estimation Approach can usefully be incorporated into conventional methods. However, the prevailing philosophy of the Estimation Approach does not address certain important needs of much experimental neuroscience. Neuroscience should adopt beneficial elements of the Estimation Approach without giving up the advantages of significance testing.

Differential RNA methylation analysis for MeRIP-seq data under general experimental design.

RNA epigenetics is an emerging field to study the post-transcriptional gene regulation. The dynamics of RNA epigenetic modification have been reported to associate with many human diseases. Recently developed high-throughput technology named Methylated RNA Immunoprecipitation Sequencing (MeRIP-seq) enables the transcriptome-wide profiling of N6-methyladenosine (m6A) modification and comparison of RNA epigenetic modifications. There are a few computational methods for the comparison of mRNA modifications under different conditions but they all suffer from serious limitations. In this work, we develop a novel statistical method to detect differentially methylated mRNA regions from MeRIP-seq data. We model the sequence count data by a hierarchical negative binomial model that accounts for various sources of variations and derive parameter estimation and statistical testing procedures for flexible statistical inferences under general experimental designs. Extensive benchmark evaluations in simulation and real data analyses demonstrate that our method is more accurate, robust and flexible compared to existing methods. Our method TRESS is implemented as an R/Bioconductor package and is available at https://bioconductor.org/packages/devel/TRESS. Supplementary data are available at Bioinformatics online.