Unreliable Inferences Research Articles

The P-value and the problem of multiple testing

The P-value and the problem of multiple testing

Latent drop-out based transitions in linear quantile hidden Markov models for longitudinal responses with attrition

Longitudinal data are characterized by the dependence between observations from the same individual. In a regression perspective, such a dependence can be usefully ascribed to unobserved features (covariates) specific to each individual. On these grounds, random parameter models with time-constant or time-varying structure are now well established in the generalized linear model context. In the quantile regression framework, specifications based on random parameters have only recently known a flowering interest. We start from the recent proposal by Farcomeni (Stat Comput 22:141---152, 2012) on longitudinal quantile hidden Markov models, and extend it to handle potentially informative missing data mechanisms. In particular, we focus on monotone missingness which may lead to selection bias and, therefore, to unreliable inferences on model parameters. We detail the proposed approach by re-analyzing a well known dataset on the dynamics of CD4 cell counts in HIV seroconverters and by means of a simulation study reported in the supplementary material.

Mechanism-Based Markers of Drug-Induced Liver Injury to Improve the Physiological Relevance and Predictivity ofIn VitroModels

Abstract It is evident from the drug development failures of the past 30 years that only through a better mechanistic understanding of specific drug-induced liver injury (DILI) will we improve our ability to predict clinical hepatotoxicity. Current in vitro models used for the prediction of DILI often suffer from uninformative, inappropriate, and poorly translatable endpoint measurements, and this may lead to unreliable inferences being drawn. Moreover, there is currently a significant disconnect between in vitro DILI endpoints and those used to assess hepatotoxicity in a clinical setting. This perspective article aims to identify the gaps in this area and reviews up-to-date information on current approaches to apply a human context to laboratory in vitro models. In order to develop better models to predict more accurately which chemicals will elicit hepatotoxicity in humans, a coordinated research effort is required involving the pharmaceutical industry, academia, and regulatory bodies. Key to this is se...

Robust twin boosting for feature selection from high-dimensional omics data with label noise

Omics data such as microarray transcriptomic and mass spectrometry proteomic data are typically characterized by high dimensionality and relatively small sample sizes. In order to discover biomarkers for diagnosis and prognosis from omics data, feature selection has become an indispensable step to find a parsimonious set of informative features. However, many previous studies report considerable label noise in omics data, which will lead to unreliable inferences to select uninformative features. Yet, to the best of our knowledge, very few feature selection methods are proposed to address this problem. This paper proposes a novel ensemble feature selection algorithm, robust twin boosting feature selection (RTBFS), which is robust to label noise in omics data. The algorithm has been validated on an omics feature selection test bed and seven real-world heterogeneous omics datasets, of which some are known to have label noise. Compared with several state-of-the-art ensemble feature selection methods, RTBFS can select more informative features despite label noise and obtain better classification results. RTBFS is a general feature selection method and can be applied to other data with label noise. MATLAB implementation of RTBFS and sample datasets are available at: http://www.cs.bham.ac.uk/∼szh/TReBFSMatlab.zip.

The link between urban development and the modal split in commuting: the case of Biscay

This paper presents an empirical analysis of the links between different land use patterns and the modal split in commuting. The analysis is conducted at municipal level, based on data on land use and mobility characteristics of the de-urbanisation process in Biscay, a small European province, from 1991 to 2001. The links are assessed by means of multiple regression analysis, in which specific techniques of spatial econometrics are applied in order to avoid biased results and unreliable inferences due to spatial dependence. The findings reveal, at municipal level, the importance of gross population density, geographic concentration of the population and, to a lesser extent, the ratio of jobs to residents in explaining the modal split in commuting, and the influence of access to public transport. It is also shown that public transport services have spillover effects that extend beyond the limits of the municipality and encourage use in adjacent areas.

A review of recent variable selection methods in industrial and chemometrics applications

The massive amount of data collected from industrial processes has challenged researchers and practitioners, turning variable selection into a research topic of interest both in academia and in industry. The use of redundant, irrelevant, and noisy variables tends to compromise the performance of many statistical tools, leading to unreliable inferences and costly data collection. In this paper, we present a literature review on recent variable selection methods and applications in manufacturing and in the chemometrics field. These methods are deployed into two major categories: variable selection for prediction of continuous response variables and for prediction of a categorical variable (also referred to as classification). Future research directions are also outlined.

Inferring a single variable from an assemblage with multiple controls: getting into deep water with cladoceran lake-depth transfer functions

Transfer functions have proved very useful for quantitative reconstruction of past environments. Inferring values of a single parameter based on changes in a community with multiple controls may result in unreliable inferences. To assess this unreliability cladoceran surface sediment assemblages from 53 lakes in Greenland, which have substantial variations in lake depth and fish abundance, both of which shape cladoceran communities, were analysed in this study. Redundancy analysis (RDA) revealed that maximum lake depth and either fish abundance or fish presence/absence exerted substantial and significant control on the cladoceran assemblage. Partial RDA showed that maximum lake depth and fish abundance uniquely explained 7.9 and 5.1%, respectively, with 5.3% variance being shared. A transfer function to infer lake depth from cladoceran sub-fossils was constructed and performed moderately well [coefficient of determination (r 2) = 0.65; root mean square error of prediction (RMSEP) = 0.32 log maximum depth] on the full dataset. When outliers, defined by a bootstrapped prediction error greater than 25% of the total depth gradient, were excluded, the model performed well (r 2 = 0.74, RMSEP = 0.25 log maximum depth). The improved transfer function was then applied to sedimentary assemblage from a sediment core from Lake Boreso, in North-eastern Greenland, covering 9,000 years. A large increase in lake depth was inferred around 6250 bp. Whilst the climate was wetter at that time, the inferred changes in depth likely reflect the alteration of the food web, which resulted from the arrival of fish in the lake. This highlights the risks of using single-variable inference models for hindcasting change in lake physical and/or food web structure when there are other important co-variables.

Risk Premium Estimation with Multicollinear and Invariant Betas by the Two-Pass Cross-Sectional Regressions

This paper analyzes the finite-sample performance of the two-pass (TP) estimators of factor risk prices when betas have high cross-sectional correlations (Multicollinear) and when betas have small cross-sectional variations (Invariant). Our Monte Carlo simulations, calibrated using actual financial data, show that the TP estimation can lead to unreliable statistical inferences when betas are multicollinear and/or invariant. Estimates of factor risk prices and the pricing intercept can have biases larger than 100% of the true value. Also the t-test for hypothesis on risk prices and the intercept have very low statistical power in these cases. Various pre-diagnostic methods are proposed in order to evaluate the reliability of the inferences from the TP estimation in empirical work.

A Note on Bayesian Inference After Multiple Imputation

This article is aimed at practitioners who plan to use Bayesian inference on multiply-imputed datasets in settings where posterior distributions of the parameters of interest are not approximately Gaussian. We seek to steer practitioners away from a naive approach to Bayesian inference, namely estimating the posterior distribution in each completed dataset and averaging functionals of these distributions. We demonstrate that this approach results in unreliable inferences. A better approach is to mix draws from the posterior distributions from each completed dataset, and use the mixed draws to summarize the posterior distribution. Using simulations, we show that for this second approach to work well, the number of imputed datasets should be large. In particular, five to ten imputed datasets—which is the standard recommendation for multiple imputation—is generally not enough to result in reliable Bayesian inferences.

Akaike-type criteria and the reliability of inference: Model selection versus statistical model specification

Since the 1990s, the Akaike Information Criterion ( AIC) and its various modifications/extensions, including BIC, have found wide applicability in econometrics as objective procedures that can be used to select parsimonious statistical models. The aim of this paper is to argue that these model selection procedures invariably give rise to unreliable inferences, primarily because their choice within a prespecified family of models (a) assumes away the problem of model validation, and (b) ignores the relevant error probabilities. This paper argues for a return to the original statistical model specification problem, as envisaged by Fisher (1922), where the task is understood as one of selecting a statistical model in such a way as to render the particular data a truly typical realization of the stochastic process specified by the model in question. The key to addressing this problem is to replace trading goodness-of-fit against parsimony with statistical adequacy as the sole criterion for when a fitted model accounts for the regularities in the data.

Spatial modelling of left censored water quality data

AbstractEnvironmental monitoring data is often spatially correlated and left censored. In this paper a previously proposed Bayesian approach to handling spatially correlated data is modified so that bias corrected estimates of variance and spatial correlation parameters are attained. The methodology is applied to a water quality data set from the Ecosystem Health Monitoring Program (EHMP) in south‐east Queensland, and the results are contrasted with those from uncorrected for bias variance estimates to show that the latter can lead to unreliable inferences. A simulation study is conducted which shows that the bias corrected estimates of variance and correlation parameters are less biased than uncorrected estimates of these parameters and that the credible intervals for the parameters from bias corrected analyses are wider than those from the uncorrected analyses. The simulation also suggests that predictions of below detection values are generally overestimated by both bias corrected and uncorrected analyses, but the latter predictions are more biased. For predictions of detectable concentrations the simulations suggest that bias corrected and uncorrected analyses are equally biased and both underestimate the true values. Copyright © 2009 John Wiley & Sons, Ltd.

Testing Uncovered Interest Parity: A Continuous-Time Approach

Nowadays researchers can choose the sampling frequency of exchange rates and interest rates. If the number of observations per contract period is large relative to the sample size, standard GMM asymptotic theory provides unreliable inferences in UIP regression tests. We specify a bivariate continuous-time model for exchange rates and forward premia robust to temporal aggregation, unlike the discrete time models in the literature. We obtain the UIP restrictions on the continuous-time model parameters, which we estimate efficiently, and propose a novel specification test that compares estimators at different frequencies. Our empirical results based on correctly specified models reject UIP.

Evaluating Scientific Inferences about the Florida Panther

At the request of the U.S. Fish and Wildlife Service (USFWS) and the Florida Fish and Wildlife Conservation Commission (FWC), we provide an independent assessment of the reliability of the scientific literature used to support conservation of Florida panthers (Puma concolor coryi). We independently reached similar conclusions about unreliable scientific inferences before discussing the issues with each other or with others. Although a quarter-century of research supports many published conclusions, 2 sets of unreliable inferences may compromise efforts to conserve the species. The first is a set of 4 unreliable inferences that underlie the Panther Habitat Evaluation Model (PHEM), used by agencies to evaluate projects that may affect panther habitat. Specifically, the following assertions are unreliable: 1) panthers are forest obligates, 2) panthers require large (>500 ha) forest patches, 3) panthers are reluctant to cross 90-m gaps of nonforest habitat, and 4) the value of potential panther habitat declines linearly with distance to a population core in south Florida, USA. These assertions are unreliable because the analyses excluded (without mention or rationale) almost half the available data, compared used habitats to an inappropriate set of available habitats, made inferences about habitat preference without any data on available habitats, were based only on panther locations during daytime, ignored telemetry error, or suffered from other flaws. The second is a set of 2 unreliable inferences about panther demography prior to the genetic restoration effort initiated in 1995. Inferences that neonate survival was ≥0.84 and that the panther population was demographically vigorous prior to 1995 are flawed because the survival analysis ignored mortality during the first 4 months and because other inferences were based on numbers of births and deaths in samples of convenience rather than appropriate vital rates. These faulty inferences about panther demography brought unwarranted credibility to challenges of the genetic restoration program. Faulty inferences of both sets were repeated in subsequent scientific and popular articles; in several instances, previously published work was mis-cited. In its current (2002–2005) version, PHEM is unreliable and should not be used in decisions about panther habitat. Biologists should obtain better demographic estimates and fully analyze how the introgression program has affected these rates.

Improving The Use Of Science In Conservation: Lessons From The Florida Panther

In a companion article (Beier et al. 2006), we identified 2 sets of unreliable inferences that may compromise efforts to conserve the Florida panther (Puma concolor coryi). In spite of serious flaws in methodology and interpretation, these unreliable conclusions have appeared in prominent, peer-refereed scientific journals and have been repeatedly cited and miscited in support of panther conservation. Future editors and referees may reduce these errors by insisting on adherence to an Introduction, Methods, Results, and Discussion (IMRAD) format; checking improbable assertions attributed to earlier papers; and refusing to allow scientific inference in publication formats not subject to scientific peer review (e.g., editorials). We urge conservation biologists to view science as an adaptive process and to use the method of multiple working hypotheses (Chamberlin 1890) that are now a central feature of adaptive resource management (Walters 1986, Williams et al. 2002). We advocate a workshop approach, similar to that used for analysis of data for the northern spotted owl (Strix occidentalis; Anderson et al. 1999), to deal with scientific disagreement where, as in the case with panthers, stakeholders have entrenched points of view. Finally, we recommend the creation of an independent Scientific Steering Committee to address long-term issues of future research and monitoring of Florida panthers.

Analysis of the impact of prawn trawling on benthic species in the Great Barrier Reef Marine Park

AbstractPrawn trawling is a major fishing activity in the Great Barrier Reef Marine Park, generating millions of dollars for the Australian economy. It is important that appropriate management decisions are made to ensure the sustainability of the industry. An experimental study was conducted to quantify the recovery of seabed fauna after repeated trawling. Data from the study were composed of counts of observed benthic organisms on the seabed. The data were analysed using models based on a generalization of the simple Poisson process, which allows for over‐dispersion and under‐dispersion relative to the Poisson distribution. It was found that these models more faithfully described the data than more standard models such as the Poisson or negative binomial. It is argued that the use of inappropriate models can result in unreliable inferences, which in turn could lead to poor management decisions. Copyright © 2004 John Wiley & Sons, Ltd.

Advances in the Analysis of Non-stationary Time Series: An Illustration of Cointegration and Error Correction Methods in Research on Crime and Immigration

Time series data of interest to social scientists often have the property of random walks in which the statistical properties of the series including means and variances vary over time. Such non-stationary series are by definition unpredictable. Failure to meet the assumption of stationarity in the process of analyzing time series variables may result in spurious and unreliable statistical inferences. This paper outlines the problems of using non-stationary data in regression analysis and identifies innovative solutions developed recently in econometrics. Cointegration and error-correction models have recently received positive attention as remedies to the problems of ``spurious regression'' arising from non-stationary series. In this paper, we illustrate the relevant statistical concepts concerning these methods by referring to similar concepts used in cross-sectional analysis. An historical example is used to demonstrate how such techniques are applied. It illustrates that ``foreign'' immigrants to Canada (1896–1940) experienced elevated levels of social control in areas of high police discretion. ``Foreign'' immigration was unrelated to trends in serious crimes but closely related to vagrancy and drunkenness. The merits of cointegration are compared to traditional approaches to the regression analysis of time series.

Male mating strategies: models of roving and residence

Efforts are on to recover tiger populations range-wide, but we lack suitable metrics to characterise and evaluate such recoveries. Identifying such metrics requires an understanding of tiger population dynamics and its ecological correlates in either recovering populations or those exposed to anthropogenic influences. We monitored a recovering tiger population from 2004 to 2017 in Rajaji National Park, India. Using photographic data and spatial capture-recapture models in an inviolate site vacated by pastoralists (zone 1), we identified demographic parameters of a recovering population. By contrasting it with a site that is presently occupied by pastoralists (zone 3) and one that is isolated (zone 3), we identified conditions facilitating recovery. In zone 1, connected to a large source population in Corbett Tiger Reserve, density increased from 2.08 tigers/100 km2 to 7.07 tigers/100 km2 corresponding to an annual growth rate of 4.5%. Density also increased in zone 2 (2.6 tigers/100 km2 to 6.22 tigers/100 km2), but estimated apparent survival was 0.47 against 0.81 in zone 1. Recovery in zone 1 was accompanied by increased survival of females, while female tenure was shorter in zone 2. Due to the lack of functional connectivity, tigers in zone 3 are facing local extinction. Our results demonstrate that creating inviolate spaces to secure breeding populations and maintaining landscape-wide connectivity to expand breeding cores is critical for recovery. We highlight that relying solely on population increase may lead to unreliable inferences about population performance and instead suggest tracking survival and female land tenure to qualify recovery and success of conservation interventions.

Vector autoregression or simultaneous equations model? The intraday relationship between index arbitrage and market volatility

We conduct simulations to compare the vector autoregression (VAR) and simultaneous equations models (SEM) for examining temporal relationships among arbitrage spreads, spot price volatility, and futures price volatility. Contrary to Koch (1993), we find that the VAR model can better reveal the underlying process, and that SEM can be misleading and may yield unreliable inferences.