Statistical Issues Research Articles

Advances in Addressing Patient Heterogeneity in Economic Evaluation: A Review of the Methods Literature.

Cost-effectiveness analyses commonly use population or sample averages, which can mask key differences across subgroups and may lead to suboptimal resource allocation. Despite there being several new methods developed over the last decade, there is no recent summary of what methods are available to researchers. This review sought to identify advances in methods for addressing patient heterogeneity in economic evaluations and to provide an overview of these methods. A literature search was conducted using the Econlit, Embase and MEDLINE databases to identify studies published after 2011 (date of a previous review on this topic). Eligible studies needed to have an explicit methodological focus, related to how patient heterogeneity can be accounted for within a full economic evaluation. Sixteen studies were included in the review. Methodologies were varied and included regression techniques, model design and value of information analysis. Recent publications have applied methodologies more commonly used in other fields, such as machine learning and causal forests. Commonly noted challenges associated with considering patient heterogeneity included data availability (e.g., sample size), statistical issues (e.g., risk of false positives) and practical factors (e.g., computation time). A range of methods are available to address patient heterogeneity in economic evaluation, with relevant methods differing according to research question, scope of the economic evaluation and data availability. Researchers need to be aware of the challenges associated with addressing patient heterogeneity (e.g., data availability) to ensure findings are meaningful and robust. Future research is needed to assess whether and how methods are being applied in practice.

Guidelines for conducting, analyzing, and interpreting electrical penetration graph (EPG) experiments on herbivorous piercing–sucking insects

AbstractThe electrical penetration graph (EPG) allows researchers to obtain information on the feeding behavior of herbivorous piercing–sucking insects, mostly Hemiptera. Guidelines are proposed for designing and conducting EPG experiments on insect–plant interactions, and for analyzing and interpreting the data. Several statistical issues common in EPG experiments are discussed. Pre‐recording insect handling, wiring, and being tethered can affect feeding behavior and recommendations are made to minimize such stresses. Recording duration can have a large effect on some variables and interpretation of the data. The pros and cons of long vs. short recording duration are discussed, and suggestions are made for reconciling these conflicting concerns. Replicates lacking a specific waveform pose problems for analyzing variables associated with that waveform. Proper ways for handling these replicates when analyzing data are discussed. The waveform in progress when the recording ends will be terminated before it would naturally end; consequently, its duration will be artificially short (truncated). This creates a problem for calculating mean or median duration of waveform periods (a waveform period is a single occurrence of the waveform), especially for waveforms that tend to occur relatively few times during a recording and in long periods, such as waveforms E2 and G (phloem and xylem sap ingestion for aphids) or Xi (xylem sap ingestion for xylem feeders). Based on a detailed analysis (see the supporting information), guidance is given on how to calculate mean and median duration of waveform periods when the last period is truncated. A modified experimental design is suggested to avoid the problem of truncated waveform periods. Most EPG variables are interdependent, and their values can affect one another. This interdependence should be considered when drawing conclusions from the data. In studies on plant resistance against vascular sap‐feeders, variables that are associated with resistance encountered during the penetration route to the vascular tissue and those encountered in the vascular tissue are discussed.

Robust nonparametric regression based on deep ReLU neural networks

In this paper, we consider robust nonparametric regression using deep neural networks with ReLU activation function. While several existing theoretically justified methods are geared towards robustness against identical heavy-tailed noise distributions, the rise of adversarial attacks has emphasized the importance of safeguarding estimation procedures against systematic contamination. We approach this statistical issue by shifting our focus towards estimating conditional distributions. To address it robustly, we introduce a novel estimation procedure based on ℓ-estimation. Under a mild model assumption, we establish general non-asymptotic risk bounds for the resulting estimators, showcasing their robustness against contamination, outliers, and model misspecification. We then delve into the application of our approach using deep ReLU neural networks. When the model is well-specified and the regression function belongs to an α-Hölder class, employing ℓ-type estimation on suitable networks enables the resulting estimators to achieve the minimax optimal rate of convergence. Additionally, we demonstrate that deep ℓ-type estimators can circumvent the curse of dimensionality by assuming the regression function closely resembles the composition of several Hölder functions. To attain this, new deep fully-connected ReLU neural networks have been designed to approximate this composition class. This approximation result can be of independent interest.

Statistical inference for generative adversarial networks and other minimax problems

AbstractThis paper studies generative adversarial networks (GANs) from the perspective of statistical inference. A GAN is a popular machine learning method in which the parameters of two neural networks, a generator and a discriminator, are estimated to solve a particular minimax problem. This minimax problem typically has a multitude of solutions and the focus of this paper are the statistical properties of these solutions. We address two key statistical issues for the generator and discriminator network parameters, consistent estimation and confidence sets. We first show that the set of solutions to the sample GAN problem is a (Hausdorff) consistent estimator of the set of solutions to the corresponding population GAN problem. We then devise a computationally intensive procedure to form confidence sets and show that these sets contain the population GAN solutions with the desired coverage probability. Small numerical experiments and a Monte Carlo study illustrate our results and verify our theoretical findings. We also show that our results apply in general minimax problems that may be nonconvex, nonconcave, and have multiple solutions.

Understanding the alternative Mantel-Haenszel statistic: Factors affecting its robustness to detect non-uniform DIF

Test-item bias has become an increasingly challenging issue in statistics and education. A popular method, the Mantel-Haenszel test, is used for detecting non-uniform differential item functioning (DIF) but requires constructing several performance tiers to maintain robustness. The alternative Mantel-Haenszel (AMH) test was developed within the last three decades as a proxy procedure requiring only two scoring tiers. However, there needs to be more information on how essential factors like comparison group sizes and question discrimination affect its ability to detect bias. This study investigates how item difficulty and discrimination and the examined ratio between the focal and reference groups impact the likelihood of the AMH test detecting DIF. A comprehensive simulation study was conducted where the test scores were generated under three conditions. There are three commonly used difficulty levels (easy, medium, and hard), two discrimination levels (referred to as 'low’ and 'high’), and three group comparison ratios (1:1, 2:1, and 5:1). The simulation study showed the AMH test’s detection rates comparable to those of other standardized procedures like the Breslow-Day (BD) and even better. The current study aims to investigate and determine the factors that affect AMH test detection behavior. The study concludes with an application involving collegiate-level test data comparing students across genders and majors.

Consumer Price Index in the Czech Republic – New Sources and Data Processing

Consumer price index has been in the centre of interest for many years, since being published in 1990s in the Czech Republic but recent price growth raised more questions on methodology and data sources used in price statistics. Users are interested not only in the figures itself but also in statistical issue influencing interpretation and the quality of consumer price index that is often used as an approximation of inflation rate. The paper introduces price statistics compiled by the Czech Statistical Office and it specifically focuses on data sources and in particular scanner data. The paper explains how advanced statistical methods such as machine learning are implemented in official statistical production. We think that the official statistics is being on the historical junction where modern methods are going to be implemented. Our paper shows the usage of machine learning procedures applied on scanner data within consumer price index. Used method is based on logistic regression and powerful Python solution and that provides fast and high quality results.

A critical evaluation and research agenda for the study of psychological dispositions and political attitudes

AbstractPolitical psychologists often examine the influence of psychological dispositions on political attitudes. Central to this field is the ideological asymmetry hypothesis (IAH), which asserts significant psychological differences between conservatives and liberals. According to the IAH, conservatives tend to exhibit greater resistance to change, a stronger inclination to uphold existing social systems, and heightened sensitivity to threats and uncertainty compared with their liberal counterparts. Our review and reanalysis, however, question the empirical strength of the IAH. We expose major concerns regarding the construct validity of the psychological dispositions and political attitudes traditionally measured. Furthermore, our research reveals that the internal validity of these studies is often compromised by endogeneity and selection biases. External and statistical validity issues are also evident, with many findings relying on small effect sizes derived from nonrepresentative student populations. Collectively, these data offer scant support for the IAH, indicating that simply amassing similar data is unlikely to clarify the validity of the hypothesis. We suggest a more intricate causal model that addresses the intricate dynamics between psychological dispositions and political attitudes. This model considers the bidirectional nature of these relationships and the moderating roles of individual and situational variables. In conclusion, we call for developing more sophisticated theories and rigorous research methodologies to enhance our comprehension of the psychological underpinnings of political ideology.

Reconsidering Sampling and Costs for Face-to-Face Surveys in the 21st Century

Abstract Some fundamental factors important for designing multistage face-to-face household surveys in the United States have not been reconsidered since they were established in the 1940s despite vast changes in all aspects of conducting these surveys. This research examines how some changes, particularly data sources and data collection costs, might affect sample designs. Modern data collection methods and tools, address-based household lists, multimode designs, and training and management methods are critical components in the cost structure of today’s surveys. We explore a sample design that uses Census tracts at the first stage rather than counties as traditionally used. The tract design then samples households directly without the additional stages of subsampling of segments in the traditional design. We present statistical and operational issues associated with the designs and discuss the cost implications. Our examination finds that, for household surveys in the United States that include face-to-face data collection, tract designs that have twice as many sampled first-stage units (but the same overall sample size) have roughly the same cost as the county design, with effective sample sizes in the range of about 25 percent higher to as much as four times higher than the county design.

Assessing bias in GFR estimating equations: improper GFR stratification can yield misleading results.

Assessing bias (estimated - measured) is key to evaluating glomerular filtration rate (GFR). Stratification by subgroups can indicate where equations perform differently. However, there is a fallacy in the assessment of two instruments (e.g., eGFR and mGFR) when stratifying on the level of only one of those instruments. Here, we present statistical aspects of the problem and a solution for GFR stratification along with an empirical investigation using data from the CKiD study. Compared and contrasted biases (eGFR relative to mGFR) with 95% confidence intervals within strata of mGFR only, eGFR only, and the average of mGFR and eGFR using data from the Chronic Kidney Disease in Children (CKiD) study. A total of 304 participants contributed 843 GFR studies with a mean mGFR of 48.46 (SD = 22.72) and mean eGFR of 48.67 (SD = 22.32) and correlation of 0.904. Despite strong agreement, eGFR significantly overestimated mGFR when mGFR < 30 (+ 6.2%; 95%CI + 2.9%, + 9.7%) and significantly underestimated when mGFR > 90 (-12.2%; 95%CI - 17.3%, - 7.0%). Significant biases in opposite direction were present when stratifying by eGFR only. In contrast, when stratifying by the average of eGFR and mGFR, biases were not significant (+ 1.3% and - 1.0%, respectively) congruent with strong agreement. Stratifying by either mGFR or eGFR only to assess eGFR biases is ubiquitous but can lead to inappropriate inference due to intrinsic statistical issues that we characterize and empirically illustrate using data from the CKiD study. Using the average of eGFR and mGFR is recommended for valid inferences in evaluations of eGFR biases.

Integrating data from different taxonomic resolutions to better estimate community alpha diversity

Integrated distribution models (IDMs), in which datasets with different properties are analysed together, are becoming widely used to model species distributions and abundance in space and time. To date, the IDM literature has focused on technical and statistical issues, such as the precision of parameter estimates and mitigation of biases arising from unstructured data sources. However, IDMs have an unrealised potential to estimate ecological properties that could not be properly derived from the source datasets if analysed separately. We present a model that estimates community alpha diversity metrics by integrating one species‐level dataset of presence–absence records with a co‐located dataset of group‐level counts (i.e. lacking information about species identity). We illustrate the ability of community IDMs to capture the true alpha diversity through simulation studies and apply the model to data from the UK Pollinator Monitoring Scheme, to describe spatial variation in the diversity of solitary bees, bumblebees and hoverflies. The simulation and case studies showed that the proposed IDM produced more precise estimates of the community diversity than the single models, and the analysis of the real dataset further showed that the alpha diversity estimates from the IDM were averages of the single models. Our findings also revealed that IDMs had a higher prediction accuracy for all the insect groups in most cases, with this performance linked to the information provided by a data source into the IDM.

A Unified Framework for Tuning Hyperparameters in Clustering Problems

Selecting hyperparameters for unsupervised learning problems is challenging in general due to the lack of ground truth for validation. Despite the prevalence of this issue in statistics and machine learning, especially in clustering problems, there are not many methods for tuning these hyperparameters with theoretical guarantees. In this paper, we provide a framework with provable guarantees for selecting hyperparameters in a number of distinct models. We consider both the subgaussian mixture model and network models to serve as examples of i.i.d. and non-i.i.d. data. We demonstrate that the same framework can be used to choose the Lagrange multipliers of penalty terms in semi-definite programming (SDP) relaxations for community detection, and the bandwidth parameter for constructing kernel similarity matrices for spectral clustering. By incorporating a cross-validation procedure, we show the framework can also do consistent model selection for network models. Using a variety of simulated and real data examples, we show that our framework outperforms other widely used tuning procedures in a broad range of parameter settings.

Management of second ipsilateral breast tumor event: An advocacy for a randomized trial

For a second ipsilateral breast tumor event, salvage mastectomy is the standard of care while second conservative treatment is a possible option. However, level 1 proofs are missing, leading to perform salvage mastectomy for patients who could receive second conservative treatment and consequently avoid psychological/quality of life salvage mastectomy deleterious impacts. A phase 3 randomized trial comparing salvage mastectomy to second conservative treatment is needed. Here we discuss what would be to us the optimal design of such trial to confirm the non-inferiority between the two salvage options, with a focus on methodological aspects in terms of patient characteristics and statistical issues.

Частный музей в пространстве культуры: проблемы определения и статистики

Private museums play an important role in the modern cultural space of Russia; their activities make a significant contribution to the preservation and actualization of cultural heritage. The study of private museums as a special typological group that developed in the post-Soviet period has a number of terminological and statistical issues. In the context of the legal insecurity of the name “museum” and the lack of licensing of museum activities, many private organizations call themselves museums, but at the same time they do not meet the definition of a museum in the Russian legislation and the definition of the International Council of Museums (ICOM). Despite carrying out some functions of a museum and practicing some types of activities characteristic of museums, they are not museums, but museum-type institutions — creative workshops, recreational or play centers, shops, etc. There is no term “private museum” in the legislation, and the measurement uncertainty of this category of institutions remains in the statistics. An analysis of the statistical reports of a number of European countries shows that the category of “private museum” in different countries has different content. In Russia, due to the peculiarities of conducting statistical monitoring of the museum sphere, private museums are not taken into account by means of state statistics. The problem of terminological uncertainty of a private museum also exists in museology: there is no single interpretation of this group of institutions, and its specific features and essential characteristics need further study and definition.

Statistical inference with regularized optimal transport

Abstract Optimal transport (OT) is a versatile framework for comparing probability measures, with many applications to statistics, machine learning and applied mathematics. However, OT distances suffer from computational and statistical scalability issues to high dimensions, which motivated the study of regularized OT methods like slicing, smoothing and entropic penalty. This work establishes a unified framework for deriving limit distributions of empirical regularized OT distances, semiparametric efficiency of the plug-in empirical estimator and bootstrap consistency. We apply the unified framework to provide a comprehensive statistical treatment of (i) average- and max-sliced $p$-Wasserstein distances, for which several gaps in existing literature are closed; (ii) smooth distances with compactly supported kernels, the analysis of which is motivated by computational considerations; and (iii) entropic OT, for which our method generalizes existing limit distribution results and establishes, for the first time, efficiency and bootstrap consistency. While our focus is on these three regularized OT distances as applications, the flexibility of the proposed framework renders it applicable to broad classes of functionals beyond these examples.

Investigation of statistical methods used in prognostic prediction models for obstetric care: A 10 year-span cross-sectional study.

Obstetric care is a highly active area in the development and application of prognostic prediction models. The development and validation of these models often require the utilization of advanced statistical techniques. However, failure to adhere to rigorous methodological standards could greatly undermine the reliability and trustworthiness of the resultant models. Consequently, the aim of our study was to examine the current statistical practices employed in obstetric care and offer recommendations to enhance the utilization of statistical methods in the development of prognostic prediction models. We conducted a cross-sectional survey using a sample of studies developing or validating prognostic prediction models for obstetric care published in a 10-year span (2011-2020). A structured questionnaire was developed to investigate the statistical issues in five domains, including model derivation (predictor selection and algorithm development), model validation (internal and external), model performance, model presentation, and risk threshold setting. On the ground of survey results and existing guidelines, a list of recommendations for statistical methods in prognostic models was developed. A total of 112 eligible studies were included, with 107 reporting model development and five exclusively reporting external validation. During model development, 58.9% of the studies did not include any form of validation. Of these, 46.4% used stepwise regression in a crude manner for predictor selection, while two-thirds made decisions on retaining or dropping candidate predictors solely based on p-values. Additionally, 26.2% transformed continuous predictors into categorical variables, and 80.4% did not consider nonlinear relationships between predictors and outcomes. Surprisingly, 94.4% of the studies did not examine the correlation between predictors. Moreover, 47.1% of the studies did not compare population characteristics between the development and external validation datasets, and only one-fifth evaluated both discrimination and calibration. Furthermore, 53.6% of the studies did not clearly present the model, and less than half established a risk threshold to define risk categories. In light of these findings, 10 recommendations were formulated to promote the appropriate use of statistical methods. The use of statistical methods is not yet optimal. Ten recommendations were offered to assist the statistical methods of prognostic prediction models in obstetric care.

U.S. stillbirth surveillance: The national fetal death file and other data sources

The National Vital Statistics System is the primary source of information on fetal deaths of 20 weeks of gestation or more in the United States. Data are cooperatively produced by jurisdiction vital statistics offices and the National Center for Health Statistics. In order to promote the uniformity of data, the National Center for Health Statistics issues The Model State Vital Statistics Act and Regulations, and produces standard certificates and reports, developed in collaboration with the states, to inform the development of jurisdictional vital records laws and regulations and data collection. While there are challenges in collecting national fetal death data, there are ongoing data quality improvement efforts to address them. Improved national fetal death data and data from other sources will continue to add insights into the risks, causes and prevention of fetal death.

Some statistical problems involved in forecasting and estimating the spread of SARS-CoV-2 using Hawkes point processes and SEIR models

This article reviews some of the statistical issues involved with modeling SARS-CoV02 (Covid-19) in Los Angeles County, California, using Hawkes point process models and SEIR models. The two types of models are compared, and their pros and cons are discussed. We also discuss particular statistical decisions, such as where to place the upper limits on y-axes, and whether to use a Bayesian or frequentist version of the model, how to estimate seroprevalence, and fitting the density of transmission times in the Hawkes model.

Which bird traits most affect the goodness-of-fit of species distribution models?

Species distribution models (SDMs) are numerical tools that combine species occurrence (or abundance) data with environmental variables, to predict the species’ distribution spatially. SDMs are increasingly used for purposes of conservation planning and management of ecosystems. The model performance can be measured as the goodness-of-fit (GOF), which describes how well it fits (e.g., the discrepancy between the statistical model and the data observed). However, there is still a need for a deeper understanding of the ecological characteristics of the modelled species which can affect the accuracy of those models. Here, we compared the goodness-of-fit of SDMs, considering several ecological characteristics of 56 bird species: Most frequently used environment, body mass, home-range, species specialization index (SSI), diet specialization and detectability. All SDMs were performed on the same dataset, and the relative frequency of each species was also incorporated to account for occurrence heterogeneity. GOF of SDMs was not significantly correlated with species’ frequency, home-range, body mass, degree of detectability or level of diet specialization. Overall, the birds with more accurate SDMs (GOF) were species of grasslands and the GOF was positively associated with SSI, indicating that more habitat-specialized species are better predictable. Our findings suggest that is important to focus not only on statistical issues potentially related to model performance but also on ecological characteristics of single species because can improve the performance of modellistic procedures, increasing their predictive power.

Statistical and Scientific Considerations Concerning the Interpretation, Replicability, and Transportability of Research Findings.

To advance scientific understanding of disease processes and related intervention effects, study results should be free from bias and replicable. More broadly, investigators seek results that are transportable, that is, applicable to a perceived study population as well as in other environments and populations. We review fundamental statistical issues that arise in the analysis of observational data from disease cohorts and other sources and discuss how these issues affect the transportability and replicability of research results. Much of the literature focuses on estimating average exposure or intervention effects at the population level, but we argue for more nuanced analyses of conditional effects that reflect the complexity of disease processes.

Commentary: On measurement error, PSA doubling time, and prostate cancer

Exposure measurement error is a pervasive problem for epidemiology research projects designed to provide valid and precise statistical evidence supporting postulated exposure-disease relationships of interest. The purpose of this commentary is to highlight an important real-life example of this exposure measurement error problem and to provide a simple and useful diagnostic tool for physicians and their patients that corrects for the exposure measurement error. More specifically, prostate-specific antigen doubling time (PSADT) is a widely used measure for guiding future treatment options for patients with biochemically recurrent prostate cancer. Numerous papers have been published claiming that a low calculated PSADT value (denoted PSADT̂) is predictive of metastasis and premature death from prostate cancer. Unfortunately, none of these papers have adjusted for the measurement error in PSADT̂, an estimator that is typically computed using the popular Memorial Sloan Kettering website very often visited by both physicians and their patients. For this website, the estimator PSADT̂ of the true (but unknown) PSADT for a patient (denoted PSADT∗) is computed as the natural log of 2 (i.e., 0.6931) divided by the estimated slope of the straight-line regression of the natural log of PSA (in ng/mL) on time. We utilize PSADT̂ to derive an expression for the probability that the unknown PSADT∗ for a patient is below a specified value C (>0) of concern to both the physician and the patient. This probability is easy to interpret and takes into account the fact that PSADT̂ is a statistical estimator with variability. This variability introduces measurement error, namely, the difference between a computed value PSADT̂ and the true, but unknown, value PSADT∗. We have developed an Excel calculator that, once the [time, ln(PSA)] values are entered, outputs both the value of PSADT̂ and the desired probability. In addition, we discuss problematic statistical issues attendant with PSADT∗ estimation typically based on at most three or four PSA values. We strongly recommend the use of this probability when physicians are discussing PSADT̂ values and associated treatment options with their patients. And, we stress that future epidemiology research projects involving PSA doubling time should take into account the measurement error problem highlighted in this Commentary.