Flawed Inference Research Articles

Accurately estimating correlations between demographic parameters: A comment on Deane etal. (2023).

Estimating correlations among demographic parameters is an important method in population ecology. A recent paper by Deane etal. (Ecology and Evolution 13:e9847, 2023) attempted to explore the effects of different priors for covariance matrices on inference when using mark-recovery data. Unfortunately, Deane etal. (2023) made a mistake when parameterizing some of their models. Rather than exploring the effects of different priors, they examined the effects of the use of incorrect equations on inference. In this manuscript, we clearly describe the mistake in Deane etal. (2023). We then demonstrate the use of an alternative and appropriate method and reach different conclusions regarding the effects of priors on inference. Consistent with other recent literature, informative inverse Wishart priors can lead to flawed inference, while vague priors on covariance matrix components have little impact when sample sizes are adequate.

Time-varying associations of patient and tumor characteristics with cancer survival: an analysis of SEER data across 14 cancer sites, 2004-2017.

Surveillance, Epidemiology, and End Results (SEER) cancer registries provides information about survival duration and cause of death for cancer patients. Baseline demographic and tumor characteristics such as age, sex, race, year of diagnosis, and tumor stage can inform the expected survival time of patients, but their associations with survival may not be constant over the post-diagnosis period. Using SEER data, we examined if there were time-varying associations of patient and tumor characteristics on survival, and we assessed how these relationships differed across 14 cancer sites. Standard Cox proportional hazards models were extended to allow for time-varying associations and incorporated into a competing-risks framework, separately modeling cancer-specific and other-cause deaths. For each cancer site and for each of the five factors, we estimated the relative hazard ratio and absolute hazard over time in the presence of competing risks. Our comprehensive consideration of patient and tumor characteristics when estimating time-varying hazards showed that the associations of age, tumor stage at diagnosis, and race/ethnicity with risk of death (cancer-specific and other-cause) change over time for many cancers; characteristics of sex and year of diagnosis exhibit some time-varying patterns as well. Stage at diagnosis had the largest associations with survival. These findings suggest that proportional hazards assumptions are often violated when examining patient characteristics on cancer survival post-diagnosis. We discuss several interesting results where the relative hazards are time-varying and suggest possible interpretations. Based on the time-varying associations of several important covariates on survival after cancer diagnosis using a pan-cancer approach, the likelihood of the proportional hazards assumption being met or corresponding interpretation should be considered in survival analyses, as flawed inference may have implications for cancer care and policy.

Robust Testing of Paired Outcomes Incorporating Covariate Effects in Clustered Data with Informative Cluster Size

Paired outcomes are common in correlated clustered data where the main aim is to compare the distributions of the outcomes in a pair. In such clustered paired data, informative cluster sizes can occur when the number of pairs in a cluster (i.e., a cluster size) is correlated to the paired outcomes or the paired differences. There have been some attempts to develop robust rank-based tests for comparing paired outcomes in such complex clustered data. Most of these existing rank tests developed for paired outcomes in clustered data compare the marginal distributions in a pair and ignore any covariate effect on the outcomes. However, when potentially important covariate data is available in observational studies, ignoring these covariate effects on the outcomes can result in a flawed inference. In this article, using rank based weighted estimating equations, we propose a robust procedure for covariate effect adjusted comparison of paired outcomes in a clustered data that can also address the issue of informative cluster size. Through simulated scenarios and real-life neuroimaging data, we demonstrate the importance of considering covariate effects during paired testing and robust performances of our proposed method in covariate adjusted paired comparisons in complex clustered data settings.

A hierarchical model for jointly assessing ecological and anthropogenic impacts on animal demography.

The management of sustainable harvest of animal populations is of great ecological and conservation importance. Development of formal quantitative tools to estimate and mitigate the impacts of harvest on animal populations has positively impacted conservation efforts.The vast majority of existing harvest models, however, do not simultaneously estimate ecological and harvest impacts on demographic parameters and population trends. Given that the impacts of ecological drivers are often equal to or greater than the effects of harvest, and can covary with harvest, this disconnect has the potential to lead to flawed inference.In this study, we used Bayesian hierarchical models and a 43‐year capture–mark–recovery dataset from 404,241 female mallards Anas platyrhynchos released in the North American midcontinent to estimate mallard demographic parameters. Furthermore, we model the dynamics of waterfowl hunters and habitat, and the direct and indirect effects of anthropogenic and ecological processes on mallard demographic parameters.We demonstrate that density dependence, habitat conditions and harvest can simultaneously impact demographic parameters of female mallards, and discuss implications for existing and future harvest management models.Our results demonstrate the importance of controlling for multicollinearity among demographic drivers in harvest management models, and provide evidence for multiple mechanisms that lead to partial compensation of mallard harvest. We provide a novel model structure to assess these relationships that may allow for improved inference and prediction in future iterations of harvest management models across taxa.

Wolf depredation hotspots in France: Clustering analyses adjusting for livestock availability

Areas exhibiting high levels of predations on livestock generate conflicts between humans and large carnivores. Managers generally seek to identify these hotspots, in order to diagnose the causes that lead to hotspot formations and to provide financial or technical support to the involved livestock owners. When locating depredation hotspots, previous studies have not adjusted for livestock availability, making it difficult for managers to discriminate hotspots resulting from underlying livestock clusters from those due to other factors such as environmental factors. We studied hotspots of wolf depredation on sheep in France from the beginning of the natural wolf recolonisation in 1994 up to 2018. For each year, we applied the Ripley's K-function and Ripley's Kinhom to determine the general depredation spatial pattern and the Kulldorff statistic to locate depredation hotspots. We showed that omitting livestock availability in these analyses led to flawed inference about the depredation pattern, and resulted in a substantial number of unidentified hotspots, including pastoral surfaces with low sheep availability. Our methodology provides reliable information for managers to understand the depredation pattern over space and time and to allocate resources.

Distinguishing within- from between-individual effects: Howto use the within-individual centring method for quadraticpatterns.

Any average pattern observed at the population level (cross‐sectional analysis) may confound two different types of processes: some processes that occur among individuals and others that occur within individuals. Separating within‐ from among‐individual processes is critical for our understanding of ecological and evolutionary dynamics.The within‐individual centring method allows distinguishing within‐ from among‐individual processes and this method has been widely used in ecology to investigate both linear and quadratic patterns. Here we show that two alternative equations could be used for the investigation of quadratic within‐individual patterns. We explain the different assumptions and constraints of both equations. Reviewing the literature, we found that mainly one of these two equations has been used in studies investigating quadratic patterns. Yet this equation might not be the most appropriate in all circumstances leading to bias and imprecision.We show that these two alternative equations make different assumptions about the shape of the within‐individual pattern. One equation assumes that the within‐individual effect is related to an absolute process whereas the other assumes the effect arises from an individual relative process. The choice of using one equation instead of the other should depend upon the biological process investigated.Using simulations, we showed that a mismatch between the assumptions made by the equation used to analyse the data and the biological process investigated might led to flawed inference affecting output of model selection and accuracy of estimates. We stress that the equation used should be chosen carefully. We provide step by step guidelines for choosing an equation when studying quadratic pattern with the within‐individual centring approach. We encourage the use of the within‐individual centring method, promoting its relevant application for nonlinear relationships.

Statistical considerations for medication adherence research

Objective Medication non-adherence is a widespread problem and has been known to be associated with worse health outcomes and increased healthcare costs. Although many measures of adherence have been developed, their usage is not consistent across studies. Furthermore, statistical methods for analyzing adherence measures have not been rigorously evaluated. Methods Using Proportion of Days Covered (PDC), a commonly used adherence measure, we examine the variability inherent to study inclusion criteria and several variations of the PDC calculation method using a motivating data example. We illustrated via sensitivity analyses the potential for flawed inference when modeling PDC as an outcome measure. We also performed simulation studies to investigate the statistical properties of three statistical models: logistic regression, negative binomial, and ordinal logistic regression models. Results Our sensitivity analysis showed that parameter estimates can vary greatly depending on the rules for determining the study end date in calculating PDC, or the minimum number of fills in defining the cohort. In simulation studies, logistic regression had lower power than ordinal logistic and negative binomial regression models. Naivete to treatment was an important predictor of adherence and omitting it from statistical models can lead to inflated type I errors. Conclusions We discourage dichotomizing adherence data as it results in low power. The negative binomial model offers advantages in modeling adherence data, as it avoids the problematic use of a ratio in regression models. The ordinal logistic regression is robust to distributional assumptions with greater power, but naivete to treatment should be adjusted to reserve type I error rate. We also provide a recommendation for defining the observation window in calculating PDC.

Managerial Authority When Knowledge is Distributed: A Knowledge Governance Perspective

A critical knowledge governance problem concerns the consequences for the use of the authority if the knowledge that is essential in a work setting is partially unknown to the person who is to exercise authority. Is it possible to rationally direct work and activities and efficiently utilize knowledge under such conditions? Recently, many scholars have given negative answers to this question, arguing that authority relations are becoming strained by the increasingly distributed nature of knowledge in and between firms. We analyze this argument on the basis of definitions of authority and distributed knowledge. This allows us to show that - while intuitively appealing - the argument that authority cannot be an efficient coordination mechanism in the presence of distributed knowledge is at best problematic. The argument is based on the flawed inference that because the holder of authority is ignorant about some of the knowledge held by employees, he cannot rationally direct them. However, it is correct that the quality of centralized direction (planning, authority) may be compromised by distributed knowledge, leading to choices of other governance mechanisms and structures.

Democracy, Geography and Model Uncertainty

AbstractWe analyse the nature of robust determinants of differences in democracy levels across countries taking explicitly into account uncertainty in the choice of covariates and spatial spillovers. We make use of recent developments in Bayesian model averaging to assess the effect of a large number of potential factors affecting democratisation processes and account for several specifications of spatial linkages. Our results indicate that spatial spillovers are present in the data even after controlling for a large number of geographical covariates. Addressing the determinants of democracy without modelling such spillovers may lead to flawed inference about the nature of the determinants of democratisation processes. In particular, our results emphasise the role played by Muslim religion, population size, trade volumes, English language, natural resource rents, GDP per capita, being a MENA country and the incidence of armed conflicts as factors affecting democracy robustly.

Impacts of an invasive plant are fundamentally altered by a co-occurring forest disturbance.

Invasive species frequently co-occur with other disturbances, which can impact the same ecosystem functions as the invader. Yet invasion studies rarely control for the presence of these other disturbances, although their overlapping effects may influence the direction and magnitude of impacts attributed to the invader alone. Here, we ask whether controlling for the presence of a co-occurring disturbance, as well as the time since disturbance, yields different values of an invader's ecosystem effects than when these factors remain unaddressed. We used a chronosequence of six forest stands at a single site: five logged stands that each contained paired invaded-uninvaded plots of the forest understory invasive grass Microstegium vimineum, as well as one unlogged and uninvaded control stand. By controlling for the presence of both logging and invasion, we untangled the effects of each through time. We found that the co-occurring disturbance of logging can dramatically alter the measured effects of M.vimineum by amplifying, dampening, negating, or entirely reversing the direction of the invader's impacts. During its period of peak impact, logging amplified the invader's positive effect on the size of the soil microbial biomass pool by 24%, reduced the invader's positive effect on soil water holding capacity by 5%, negated the invader's positive effect on the particulate organic matter carbon pool (from a 9% increase to no significant effect), and reversed the direction of the invader's impact on net nitrogen mineralization rate from a 51% increase to a 52% decrease. Furthermore, the influence of logging on the invader's impacts was not static, but dynamic through time. The results from our site therefore demonstrate that failure to account for the impacts of a co-occurring disturbance, as well as the time since disturbance, can result in flawed inference about the nature of an invader's effects. Future research should determine how widespread such flawed inference might be among other invasive species and across different environmental contexts. To help guide such research, we describe a general framework for disentangling the overlapping effects of invasions and co-occurring disturbances through time.

Uncovering periodic patterns of space use in animal tracking data with periodograms, including a new algorithm for the Lomb-Scargle periodogram and improved randomization tests.

BackgroundPeriodicity in activity level (rest/activity cycles) is ubiquitous in nature, but whether and how these periodicities translate into periodic patterns of space use by animals is much less documented. Here we introduce an analytical protocol based on the Lomb-Scargle periodogram (LSP) to facilitate exploration of animal tracking datasets for periodic patterns. The LSP accommodates missing observations and variation in the sampling intervals of the location time series.ResultsWe describe a new, fast algorithm to compute the LSP. The gain in speed compared to other R implementations of the LSP makes it tractable to analyze long datasets (>106 records). We also give a detailed primer on periodicity analysis, focusing on the specificities of movement data. In particular, we warn against the risk of flawed inference when the sampling schedule creates artefactual periodicities and we introduce a new statistical test of periodicity that accommodates temporally autocorrelated background noise. Applying our LSP-based analytical protocol to tracking data from three species revealed that an ungulate exhibited periodicity in its movement speed but not in its locations, that a central place-foraging seabird tracked moon phase, and that the movements of a range-resident canid included a daily patrolling component that was initially masked by the stochasticity of the movements.ConclusionThe new, fast algorithm tailored for movement data analysis and now available in the R-package ctmm makes the LSP a convenient exploratory tool to detect periodic patterns in animal movement data.Electronic supplementary materialThe online version of this article (doi:10.1186/s40462-016-0084-7) contains supplementary material, which is available to authorized users.

Why Ben-Naim’s Deepest Pitfall Does Not Exist

Ben-Naim warns that there are pitfalls in pursuing Anfinsen’s thermodynamic hypothesis. We show that the deepest one of his pitfalls is not a pitfall at all and the believing pursuing the minimum of the Gibbs free energy will lead us to the native structure. The “pitfall” came from the flawed inference of Gibbs energy function and the muddiness on concepts such as thermodynamic system and the second law of thermodynamics.

Random-effects, fixed-effects and the within-between specification for clustered data in observational health studies: a simulation study.

BackgroundWhen unaccounted-for group-level characteristics affect an outcome variable, traditional linear regression is inefficient and can be biased. The random- and fixed-effects estimators (RE and FE, respectively) are two competing methods that address these problems. While each estimator controls for otherwise unaccounted-for effects, the two estimators require different assumptions. Health researchers tend to favor RE estimation, while researchers from some other disciplines tend to favor FE estimation. In addition to RE and FE, an alternative method called within-between (WB) was suggested by Mundlak in 1978, although is utilized infrequently.MethodsWe conduct a simulation study to compare RE, FE, and WB estimation across 16,200 scenarios. The scenarios vary in the number of groups, the size of the groups, within-group variation, goodness-of-fit of the model, and the degree to which the model is correctly specified. Estimator preference is determined by lowest mean squared error of the estimated marginal effect and root mean squared error of fitted values.ResultsAlthough there are scenarios when each estimator is most appropriate, the cases in which traditional RE estimation is preferred are less common. In finite samples, the WB approach outperforms both traditional estimators. The Hausman test guides the practitioner to the estimator with the smallest absolute error only 61% of the time, and in many sample sizes simply applying the WB approach produces smaller absolute errors than following the suggestion of the test.ConclusionsSpecification and estimation should be carefully considered and ultimately guided by the objective of the analysis and characteristics of the data. The WB approach has been underutilized, particularly for inference on marginal effects in small samples. Blindly applying any estimator can lead to bias, inefficiency, and flawed inference.

Response to M Krkošek, C W Revie, B Finstad and C D Todd's comment on Jackson et al . ‘Impact of L epeophtheirus salmonis infestations on migrating Atlantic salmon, S almo salar L., smolts at eight locations in Ireland with an analysis of lice‐induced marine mortality’

Response to M Krkošek, C W Revie, B Finstad and C D Todd's comment on Jackson <i>et al</i> . ‘Impact of <i> <scp>L</scp> epeophtheirus salmonis </i> infestations on migrating Atlantic salmon, <i> <scp>S</scp> almo salar </i> L., smolts at eight locations in Ireland with an analysis of lice‐induced marine mortality’

Sinks as saviors: Why flawed inference cannot assist tiger recovery

A recent study of tigers in Chitwan, Nepal (1) stirred controversy by challenging the “source-sink” approach that underlies current global tiger conservation strategies (2). The observed lack of difference in tiger density estimates inside the protected area compared with a multiple-use area outside is offered as evidence. Based on this result, the study questions the relevance of strictly protected tiger reserves involving regulation of extractive uses and relocation of human settlements. The study offers an alternate vision of sustainable, syntopic “coexistence” of tigers and humans as a solution to increasing human resource demands on tiger habitats.

Transience in the humpback whale population of New Caledonia and implications for abundance estimation

AbstractA phenomenon of transience in the humpback whale population breeding in New Caledonia has been highlighted in recent analyses. We used these data to illustrate the risk of flawed inference when transience is not properly accounted for in abundance estimation of resident populations. Transients are commonly defined as individuals that pass through the sampling area once, i.e., have a null probability of being caught again, and therefore induce heterogeneity in the detection process. The presence of transients can lead to severe bias in the estimation of abundance and we demonstrate how to correct for this feature when estimating abundance of resident populations. In New Caledonia, very different conclusions about the number of resident whales in the southern lagoon between 1999 and 2005 are obtained when the abundance estimate accounts for the transient whales. Without correction, the estimates of the abundance were up to twice as high across all years compared to the estimates of the resident population when a correction for transients had been incorporated. Having reliable population estimates when assessing the status of endangered species is essential in documenting recovery and monitoring of population trends. Therefore, we encourage researchers to account for transients when reporting abundances of resident populations.

Digging through model complexity: using hierarchical models to uncover evolutionary processes in the wild

The growing interest for studying questions in the wild requires acknowledging that eco-evolutionary processes are complex, hierarchically structured and often partially observed or with measurement error. These issues have long been ignored in evolutionary biology, which might have led to flawed inference when addressing evolutionary questions. Hierarchical modelling (HM) has been proposed as a generic statistical framework to deal with complexity in ecological data and account for uncertainty. However, to date, HM has seldom been used to investigate evolutionary mechanisms possibly underlying observed patterns. Here, we contend the HM approach offers a relevant approach for the study of eco-evolutionary processes in the wild by confronting formal theories to empirical data through proper statistical inference. Studying eco-evolutionary processes requires considering the complete and often complex life histories of organisms. We show how this can be achieved by combining sequentially all life-history components and all available sources of information through HM. We demonstrate how eco-evolutionary processes may be poorly inferred or even missed without using the full potential of HM. As a case study, we use the Atlantic salmon and data on wild marked juveniles. We assess a reaction norm for migration and two potential trade-offs for survival. Overall, HM has a great potential to address evolutionary questions and investigate important processes that could not previously be assessed in laboratory or short time-scale studies.

Trading on Preconceptions: Why World War I Was Not a Failure of Economic Interdependence

World War I is generally viewed by both advocates and critics of commercial liberal theory as the quintessential example of a failure of economic integration to maintain peace. Yet this consensus relies on both methodologically flawed inference and an incomplete accounting of the antecedents to the war. Crucially, World War I began in a weakly integrated portion of Europe with which highly integrated powers were entangled through the alliance system. Crises among the highly interdependent European powers in the decades leading up to the war were generally resolved without bloodshed. Among the less interdependent powers in Eastern Europe, however, crises regularly escalated to militarized violence. Moreover, the crises leading to the war created increased incentives for the integrated powers to strengthen commitments to their less interdependent partners. In attempting to make these alliances more credible, Western powers shifted foreign policy discretion to the very states that lacked strong economic disincentives to fight. Had globalization pervaded Eastern Europe, or if the rest of Europe had been less locked into events in the east, Europe might have avoided a “Great War.”

Trading on Preconceptions: Why World War I Was Not a Failure of Economic Interdependence

The First World War is generally viewed by both advocates and critics of commercial liberal theory as the quintessential example of a failure of economic integration to maintain peace. Yet this consensus relies both on methodologically flawed inference and an incomplete accounting of the antecedents to the war. Crucially, the war began in a weakly integrated portion of Europe with which highly integrated powers were entangled through the alliance system. Crises among the highly interdependent European powers in the decades leading up to World War I were generally resolved without bloodshed. Among the less interdependent powers in Eastern Europe, however, crises regularly escalated to militarized violence. Moreover, the crises leading to the war created increased incentives for the integrated powers to strengthen commitments to their less-interdependent partners. In attempting to make these alliances more credible, western powers shifted foreign policy discretion to the very states that lacked strong economic disincentives to fight. Had globalization pervaded Eastern Europe, or the rest of Europe been less locked into events in the East, Europe might have avoided a “Great War.”

Combining capture–recapture data and pedigree information to assess heritability of demographic parameters in the wild

Quantitative genetic analyses have been increasingly used to estimate the genetic basis of life-history traits in natural populations. Imperfect detection of individuals is inherent to studies that monitor populations in the wild, yet it is seldom accounted for by quantitative genetic studies, perhaps leading to flawed inference. To facilitate the inclusion of imperfect detection of individuals in such studies, we develop a method to estimate additive genetic variance and assess heritability for binary traits such as survival, using capture-recapture (CR) data. Our approach combines mixed-effects CR models with a threshold model to incorporate discrete data in a standard 'animal model' approach. We employ Markov chain Monte Carlo sampling in a Bayesian framework to estimate model parameters. We illustrate our approach using data from a wild population of blue tits (Cyanistes caeruleus) and present the first estimate of heritability of adult survival in the wild. In agreement with the prediction that selection should deplete additive genetic variance in fitness, we found that survival had low heritability. Because the detection process is incorporated, capture-recapture animal models (CRAM) provide unbiased quantitative genetics analyses of longitudinal data collected in the wild.