Statistical Challenges Research Articles

Quantitative evaluation of footwear evidence: Initial workflow for an end-to-end system.

In the United States, footwear examiners make decisions about the sources of crime scene shoe impressions using subjective criteria. This has raised questions about the accuracy, repeatability, reproducibility, and scientific validity of footwear examinations. Currently, most footwear examiners follow a workflow that compares a questioned and test impression with regard to outsole design, size, wear, and randomly acquired characteristics (RACs). We augment this workflow with computer algorithms and statistical analysis so as to improve in the following areas: (1) quantifying the degree of correspondence between the questioned and test impressions with respect to design, size, wear, and RACs, (2) reducing the potential for cognitive bias, and (3) providing an empirical basis for examiner conclusions by developing a reference database of case-relevant pairs of impressions containing known mated and known nonmated impressions. Our end-to-end workflow facilitates all three of these points and is directly relatable to current practice. We demonstrate the workflow, which includes obtaining and interpreting outsole pattern scores, RAC comparison scores, and final scores, on two scenarios-a pristine example (involving very high quality Everspry EverOS scanner impressions) and a mock crime scene example that more closely resembles real casework. These examples not only demonstrate the workflow but also help identify the algorithmic, computational, and statistical challenges involved in improving the system for eventual deployment in casework.

Knowledge of the Unknown

Undetected and unreported crime are among the most pressing challenges of crime statistics. Since the late 1960s and early 1970s, victim surveys and self-report surveys have helped to paint a clearer picture of the social reality of crime, and in modern criminology such tools are now state of the art. Yet despite the current boom in survey-based methods, the history of this quest to define the “reality of crime” remains largely obscure. Its roots extend much further back to an age well before the “dark figure” of crime became, at least in part, a measurable entity. This paper explores the historical context behind an emerging awareness of undiscovered crime at the turn to the 19th century. The knowledge that knowledge was deficient emerged out of practices of criminal investigation. It is considered here in the light of a perception in German territories on the threshold to a modern administration that itinerant thieves and vagabonds posed a very real danger. The paper examines methods adopted to probe uncharted realms, thereby stabilizing the veracity of this “knowledge of the unknown”.

Host-associated microbe PCR (hamPCR) enables convenient measurement of both microbial load and community composition.

The ratio of microbial population size relative to the amount of host tissue, or 'microbial load', is a fundamental metric of colonization and infection, but it cannot be directly deduced from microbial amplicon data such as 16S rRNA gene counts. Because existing methods to determine load, such as serial dilution plating, quantitative PCR, and whole metagenome sequencing add substantial cost and/or experimental burden, they are only rarely paired with amplicon sequencing. We introduce host-associated microbe PCR (hamPCR), a robust strategy to both quantify microbial load and describe interkingdom microbial community composition in a single amplicon library. We demonstrate its accuracy across multiple study systems, including nematodes and major crops, and further present a cost-saving technique to reduce host overrepresentation in the library prior to sequencing. Because hamPCR provides an accessible experimental solution to the well-known limitations and statistical challenges of compositional data, it has far-reaching potential in culture-independent microbiology.

A Dirichlet Autoregressive Model for the Analysis of Microbiota Time-Series Data

Growing interest in understanding microbiota dynamics has motivated the development of different strategies to model microbiota time series data. However, all of them must tackle the fact that the available data are high-dimensional, posing strong statistical and computational challenges. In order to address this challenge, we propose a Dirichlet autoregressive model with time-varying parameters, which can be directly adapted to explain the effect of groups of taxa, thus reducing the number of parameters estimated by maximum likelihood. A strategy has been implemented which speeds up this estimation. The usefulness of the proposed model is illustrated by application to a case study.

Multispecies coalescent and its applications to infer species phylogenies and cross-species gene flow.

Multispecies coalescent (MSC) is the extension of the single-population coalescent model to multiple species. It integrates the phylogenetic process of species divergences and the population genetic process of coalescent, and provides a powerful framework for a number of inference problems using genomic sequence data from multiple species, including estimation of species divergence times and population sizes, estimation of species trees accommodating discordant gene trees, inference of cross-species gene flow and species delimitation. In this review, we introduce the major features of the MSC model, discuss full-likelihood and heuristic methods of species tree estimation and summarize recent methodological advances in inference of cross-species gene flow. We discuss the statistical and computational challenges in the field and research directions where breakthroughs may be likely in the next few years.

Geometric Variational Inference.

Efficiently accessing the information contained in non-linear and high dimensional probability distributions remains a core challenge in modern statistics. Traditionally, estimators that go beyond point estimates are either categorized as Variational Inference (VI) or Markov-Chain Monte-Carlo (MCMC) techniques. While MCMC methods that utilize the geometric properties of continuous probability distributions to increase their efficiency have been proposed, VI methods rarely use the geometry. This work aims to fill this gap and proposes geometric Variational Inference (geoVI), a method based on Riemannian geometry and the Fisher information metric. It is used to construct a coordinate transformation that relates the Riemannian manifold associated with the metric to Euclidean space. The distribution, expressed in the coordinate system induced by the transformation, takes a particularly simple form that allows for an accurate variational approximation by a normal distribution. Furthermore, the algorithmic structure allows for an efficient implementation of geoVI which is demonstrated on multiple examples, ranging from low-dimensional illustrative ones to non-linear, hierarchical Bayesian inverse problems in thousands of dimensions.

Shocks to Hospital Occupancy and Mortality: Evidence from the 2009 H1N1 Pandemic

Existing literature suggests that hospital occupancy matters for quality of care, as measured by various patient outcomes. However, estimating the causal effect of increased hospital busyness on in-hospital mortality remains an elusive task due to statistical power challenges and the difficulty in separating shocks to occupancy from changes in patient composition. Using data from a large public hospital system in Mexico, we estimate the impact of congestion on in-hospital mortality by exploiting the shock in hospitalizations induced by the 2009 H1N1 pandemic, instrumenting hospital admissions due to acute respiratory infections (ARIs) with measures of ARI cases at nearby healthcare facilities as a proxy for the size of the local outbreak. Our instrumental-variables estimates show that a 1% increase in ARI admissions in 2009 led to a 0.25% increase in non-ARI in-hospital mortality. We show that these effects are nonlinear in the size of the local outbreak, consistent with the existence of tipping points. We further show that effects are concentrated at hospitals with limited infrastructure, suggesting that supply-side policies that improve patient assignment across hospitals and strategically increase hospital capacity could mitigate some of the negative impacts. We discuss managerial implications, suggesting that up to 25%–30% of our estimated deaths at small and non-intensive-care-unit hospitals could have been averted by reallocating patients to reduce congestion. This paper was accepted by Carri Chan, healthcare management.

Abstract 892: Application of iterative sureindependence screening to improve urinary metabolomics-based prediction of survival in colorectal cancer patients

Abstract Objective: High-throughput metabolomics assays can generate thousands of biomarker measurements and provide novel opportunities for prognostic modeling in colorectal cancer (CRC) research. The high dimensionality of metabolic data brings unforeseen statistical challenges and traditional variable selection methods may not perform well due to simultaneous challenges of computational expediency, statistical accuracy, and algorithmic stability. The Iterative Sure Independence Screening (ISIS) has demonstrated superior theoretical properties in handling such situations, and may be a viable alternative. Methods: In a prospective study of 77 newly diagnosed CRC patients (stage I-IV), pre-surgical urinary samples were analyzed on a gas chromatography-mass spectrometry platform. After exclusion of metabolites with >30% coefficient of variation, 168 metabolites remained for statistical analysis. Raw measures were processed following a standard normalization pipeline. The primary outcome was overall survival (OS) as measured from date of cancer diagnosis. In addition to metabolomics data, the predictor set included baseline clinical characteristics, such as age, sex, body mass index, tumor site, tumor stage, and receipt of neo-adjuvant and/or adjuvant treatment. We applied the ISIS method with Lasso penalty on a Cox regression model (ISIS-Lasso) to identify features associated with OS. Cox models with either Lasso regularization or backward selection were also considered as competing methods. The performance of the models was assessed through two standard performance matrices: Uno's time-dependent Area Under the Curve (tAUC) and Brier's score, both with resample-based validation. Results: Based on bootstrapped tAUC curves, we demonstrated that the screening step in ISIS can significantly improve model performance, since its predicted mean (and median) AUC are larger across clinically relevant follow-up time points (2 and 5 years after diagnosis) relative to the corresponding measures from the other two models. The prediction error based on Brier's score with 0.632+ bootstrap from ISIS-Lasso was also noticeably lower compared to the model from backward selections. Based on the ISIS-Lasso model, we identified two features, that were predictive of OS in CRC patients: tumor stage and cystine. When fixing all other clinical measures, patients with early stage (I-III) had 52% lower risk of death, compared to late stage (IV); 1 standard deviation of increase of cystine level was associated with 62% increased risk of death. Conclusion: We have demonstrated the feasibility and effectiveness of an ISIS-based method to improve selection of prognostic models derived from metabolomics data. This may be especially useful for studies with moderate sample sizes. We have identified cystine as a potentially important prognostic biomarker. Citation Format: Tengda Lin, Biljana Gigic, Kenneth Boucher, Ben Haaland, David Liesenfeld, Robert Owen, Petra Schrotz-King, Jourgen Boehm, Anita Peoples, Augustin Scalbert, Martin Schneider, Jane Figueiredo, William Grady, Christopher Li, David Shibata, Erin Siegel, Adetunji Toriola, Alexis Ulrich, Neli Ulrich, Jincheng Shen, Jennifer Ose. Application of iterative sureindependence screening to improve urinary metabolomics-based prediction of survival in colorectal cancer patients [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2021; 2021 Apr 10-15 and May 17-21. Philadelphia (PA): AACR; Cancer Res 2021;81(13_Suppl):Abstract nr 892.

Computational Topology for Biomedical Images and Data: Theory and Applications [Book Review

This is a classic textbook on computational topology. Big data or data analysis, of which topological data analysis is a major component, is a burgeoning field at the moment. A foray into “big data” quickly brings to front two of the central statistical challenges of our times— detection and classification of structure in extremely large, high-dimensional, data sets. Among the most intriguing new approaches to this challenge is topological data analysis (TDA), the primary aim of which is providing topologically informative preanalyses of data, which serve as input to more quantitative analyses at a later stage. The most enticing aspect of the book is the successful rendezvous between compactness and completeness. While the compactness does not allow it to cover the entire breadth of topics relevant to the field of computational topology, it is an excellent choice for the mandate that the book aspires to fulfill—introduction of computational topology as a potentially powerful tool for diagnostics associated with biomedical images and data. The completeness is reflected in a prudent selection, and rigorous treatment of the background mathematical concepts, essential toward building a functional data analysis pipeline based on tools from computational topology. The authors have achieved a fine balance between theory and application, and the book is a pleasure to read. The book will definitely be a frequently used and a cherished item in my personal collection. Due to the breadth in the topics covered, compactness and lucidity, it is my hope and belief that the book will quickly find itself on the shelves of students and early researchers in TDA as a must-have, yet at the same time serve as a frequent go-to reference manual for faculty and seasoned researchers.

Volatility and dynamic dependence modeling: Review, applications, and financial risk management

AbstractSince the introduction of ARCH models close to 40 years ago, a wide range of models for volatility estimation and prediction have been developed and integrated into asset allocation, financial derivative pricing, and financial risk management. Research has also been very active in extending volatility modeling to dependence modeling and in developing our understanding of risk and uncertainty in financial systems. This paper presents a review on the statistical modeling on volatility and dynamic dependence of financial returns. In addition, we present a real data example using a time‐varying copula model to estimate the dynamic dependence of stock returns. Research on volatility and dynamic dependence modeling will continue to encounter statistical and computational challenges; it is necessary to persist in dealing with the 3H (high dimension, high frequency, high complexity) paradigm in modeling.This article is categorized under:Statistical Learning and Exploratory Methods of the Data Sciences > Modeling MethodsStatistical Models > Nonlinear ModelsData: Types and Structure > Time Series, Stochastic Processes, and Functional Data

SPARK-X: non-parametric modeling enables scalable and robust detection of spatial expression patterns for large spatial transcriptomic studies

Spatial transcriptomic studies are becoming increasingly common and large, posing important statistical and computational challenges for many analytic tasks. Here, we present SPARK-X, a non-parametric method for rapid and effective detection of spatially expressed genes in large spatial transcriptomic studies. SPARK-X not only produces effective type I error control and high power but also brings orders of magnitude computational savings. We apply SPARK-X to analyze three large datasets, one of which is only analyzable by SPARK-X. In these data, SPARK-X identifies many spatially expressed genes including those that are spatially expressed within the same cell type, revealing new biological insights.

Lme4GS: An R-Package for Genomic Selection.

Genomic selection (GS) is a technology used for genetic improvement, and it has many advantages over phenotype-based selection. There are several statistical models that adequately approach the statistical challenges in GS, such as in linear mixed models (LMMs). An active area of research is the development of software for fitting LMMs mainly used to make genome-based predictions. The lme4 is the standard package for fitting linear and generalized LMMs in the R-package, but its use for genetic analysis is limited because it does not allow the correlation between individuals or groups of individuals to be defined. This article describes the new lme4GS package for R, which is focused on fitting LMMs with covariance structures defined by the user, bandwidth selection, and genomic prediction. The new package is focused on genomic prediction of the models used in GS and can fit LMMs using different variance–covariance matrices. Several examples of GS models are presented using this package as well as the analysis using real data.

Study design flaws and statistical challenges in evaluating fertility treatments.

SummaryHealth interventions should be tested before being introduced into clinical practice, to find out whether they work and whether they are harmful. However, research studies will only provide reliable answers to these questions if they are appropriately designed and analysed. But these are not trivial tasks. We review some methodological challenges that arise when evaluating fertility interventions and explain the implications for a non-statistical audience. These include flexibility in outcomes and analyses; use of surrogate outcomes instead of live birth; use of inappropriate denominators; evaluating cumulative outcomes and time to live birth; allowing each patient or couple to contribute to a research study more than once. We highlight recurring errors and present solutions. We conclude by highlighting the importance of collaboration between clinical and methodological experts, as well as people with experience of subfertility, for realising high-quality research.Lay summaryWe do research to find out whether fertility treatments are beneficial and to make sure they don’t cause harm. However, research will only provide reliable answers if it is done properly. It is not unusual for researchers to make mistakes when they are designing research studies and analysing the data that we get from them. In this review, we describe some of the mistakes people make when they do research about fertility treatments and explain how to avoid them. These include challenges which arise due to the large number of things that can be measured and reported when looking to see if fertility treatments work; failure to check whether the treatment increases the number of live births; failing to include all study participants in calculations;challenges in studies where participants may have more than one treatment attempt. We conclude by highlighting the importance of collaboration between clinical and methodological experts, as well as people with experience of fertility problems.

Inference on covariance-mean regression

In this article, we introduce a covariance-mean regression model with heterogeneous similarity matrices. It not only links the covariance of responses to heterogeneous similarity matrices induced by auxiliary information, but also establishes the relationship between the mean of responses and covariates. Under this new model setting, however, two statistical inference challenges are encountered. The first challenge is that the consistency of the covariance estimator based on the standard profile likelihood approach breaks down. Hence, we propose an adjustment and develop the Z-estimation and unconstrained/constrained ordinary least squares estimation methods. We demonstrate that the resulting estimators are consistent and asymptotically normal. The second challenge is testing the adequacy of the covariance-mean regression model comprising both the multivariate mean regression and the heterogeneous covariance matrices. Correspondingly, we introduce two diagnostic test statistics and then obtain their theoretical properties. The proposed estimators and tests are illustrated via extensive simulations and an empirical example study of the stock return comovement in the US stock market.

FlorentBédécarrats, IsabelleGuérin, FrançoisRoubaud (Eds.) Randomized Control Trials in the Field of Development: A Critical PerspectiveOxford University Press, 2020, 448 p., $100.00

FlorentBédécarrats, IsabelleGuérin, FrançoisRoubaud (Eds.) Randomized Control Trials in the Field of Development: A Critical PerspectiveOxford University Press, 2020, 448 p., $100.00

Implementation of a Generalized Additive Model (GAM) for Soybean Maturity Prediction in African Environments

Time to maturity (TTM) is an important trait in soybean breeding programs. However, soybeans are a relatively new crop in Africa. As such, TTM information for soybeans is not yet as well defined as in other major producing areas. Multi-environment trials (METs) allow breeders to analyze crop performance across diverse conditions, but also pose statistical challenges (e.g., unbalanced data). Modern statistical methods, e.g., generalized additive models (GAMs), can flexibly smooth a range of responses while retaining observations that could be lost under other approaches. We leveraged 5 years of data from an MET breeding program in Africa to identify the best geographical and seasonal variables to explain site and genotypic differences in soybean TTM. Using soybean cycle features (e.g., minimum temperature, daylength) along with trial geolocation (longitude, latitude), a GAM predicted soybean TTM within 10 days of the average observed TTM (RMSE = 10.3; x = 109 days post-planting). Furthermore, we found significant differences between cultivars (p < 0.05) in TTM sensitivity to minimum temperature and daylength. Our results show potential to advance the design of maturity systems that enhance soybean planting and breeding decisions in Africa.

The statistical approach in trial-based economic evaluations matters: get your statistics together!

BackgroundBaseline imbalances, skewed costs, the correlation between costs and effects, and missing data are statistical challenges that are often not adequately accounted for in the analysis of cost-effectiveness data. This study aims to illustrate the impact of accounting for these statistical challenges in trial-based economic evaluations.MethodsData from two trial-based economic evaluations, the REALISE and HypoAware studies, were used. In total, 14 full cost-effectiveness analyses were performed per study, in which the four statistical challenges in trial-based economic evaluations were taken into account step-by-step. Statistical approaches were compared in terms of the resulting cost and effect differences, ICERs, and probabilities of cost-effectiveness.ResultsIn the REALISE study and HypoAware study, the ICER ranged from 636,744€/QALY and 90,989€/QALY when ignoring all statistical challenges to − 7502€/QALY and 46,592€/QALY when accounting for all statistical challenges, respectively. The probabilities of the intervention being cost-effective at 0€/ QALY gained were 0.67 and 0.59 when ignoring all statistical challenges, and 0.54 and 0.27 when all of the statistical challenges were taken into account for the REALISE study and HypoAware study, respectively.ConclusionsNot accounting for baseline imbalances, skewed costs, correlated costs and effects, and missing data in trial-based economic evaluations may notably impact results. Therefore, when conducting trial-based economic evaluations, it is important to align the statistical approach with the identified statistical challenges in cost-effectiveness data. To facilitate researchers in handling statistical challenges in trial-based economic evaluations, software code is provided.

The role of statistics in the design and analysis of companion diagnostic (CDx) studies

Companion diagnostic tests are crucial in the development of precision medicine. These tests provide information that is essential for the safe and effective use of specific therapeutic products. Statistics plays a key role in the design and analysis of studies to demonstrate the safety and effectiveness of the companion diagnostics. This article can serve as an introduction to companion diagnostics for therapeutic statisticians and for diagnostic ones as well as a discussion of some of the statistical challenges. The topics include biomarker development, diagnostic performance, misclassification, prospective-retrospective validation, bridging studies, missing data, follow-on diagnostics and complex signatures.

Nonparametric inference for distribution functions with stratified samples

We consider nonparametric estimation of a distribution function when data are collected from two-phase stratified sampling without replacement. We study the inverse probability weighted empirical distribution function and propose a novel computational procedure to construct a confidence band. Two-phase sampling design induces heterogeneity across strata and dependence due to sampling without replacement. Two major statistical challenges from this design are: (1) the standard practice to approximate sampling without replacement by Bernoulli sampling leads to an incorrect coverage probability, and (2) a complicated limiting process of the proposed estimator does not allow one to analytically compute quantiles of the supremum of the limiting process nor to apply existing bootstrap methods to the proposed estimator. To address these issues, we rigorously establish the asymptotic properties of the proposed estimator and develop a simulation-based method to estimate the limiting process. The finite sample performance is evaluated through a simulation study. A Wilms tumor example is provided.

A Gibbs sampler for learning DAG: a unification for discrete and Gaussian domains

One of the major challenges in modern day statistics is to formulate models and develop inferential procedures to understand the complex multivariate relationships present in high-dimensional datasets. In this paper, we address the issue of model determination for DAGs, with respect to a given ordering of the variables, together with the corresponding parameter estimation. For this, we use a hierarchical mixture prior and develop a Gibbs sampling algorithm to carry out the posterior computations. We first focus on the Gaussian DAG models and calculate the posterior probability of being the edge between two nodes. We then extend our idea to construct a DAG for discrete data under the assumption that the data generated by discretization of the marginal distributions of a latent multivariate Gaussian distribution via a set of predetermined threshold values. Results show that the proposed method has high accuracy. The source code is available at http://bs.ipm.ac.ir/softwares/Gibbs/code.rar