Erroneous Inferences Research Articles

Natural Language Processing for Drug Discovery Knowledge Graphs: Promises and Pitfalls.

Building and analyzing knowledge graphs (KGs) to aid drug discovery is a topical area of research. A salient feature of KGs is their ability to combine many heterogeneous data sources in a format that facilitates discovering connections. The utility of KGs has been exemplified in areas such as drug repurposing, with insights made through manual exploration and modeling of the data. In this chapter, we discuss promises and pitfalls of using natural language processing (NLP) to mine "unstructured text"- typically from scientific literature- as a data source for KGs. This draws on our experience of initially parsing "structured" data sources-such as ChEMBL-as the basis for data within a KG, and then enriching or expanding upon them using NLP. The fundamental promise of NLP for KGs is the automated extraction of data from millions of documents-a task practically impossible to do via human curation alone. However, there are many potential pitfalls in NLP-KG pipelines, such as incorrect named entity recognition and ontology linking, all of which could ultimately lead to erroneous inferences and conclusions.

Same but different: The latency of a shared expectation signal interacts with stimulus attributes

Predictive coding theories assert that perceptual inference is a hierarchical process of belief updating, wherein the onset of unexpected sensory data causes so-called prediction error responses that calibrate erroneous inferences. Given the functionally specialised organisation of visual cortex, it is assumed that prediction error propagation interacts with the specific visual attribute violating an expectation. We sought to test this within the temporal domain by applying time-resolved decoding methods to electroencephalography (EEG) data evoked by contextual trajectory violations of either brightness, size, or orientation within a bound stimulus. We found that following ∼170 ms post stimulus onset, responses to both size violations and orientation violations were decodable from physically identical control trials in which no attributes were violated. These two violation types were then directly compared, with attribute-specific signalling being decoded from 265 ms. Temporal generalisation suggested that this dissociation was driven by latency shifts in shared expectation signalling between the two conditions. Using a novel temporal bias method, we then found that this shared signalling occurred earlier for size violations than orientation violations. To our knowledge, we are among the first to decode expectation violations in humans using EEG and have demonstrated a temporal dissociation in attribute-specific expectancy violations.

Interviewer Effects on the Measurement of Physical Performance in a Cross-National Biosocial Survey.

Biosocial surveys increasingly use interviewers to collect objective physical health measures (or "biomeasures") in respondents' homes. While interviewers play an important role, their high involvement can lead to unintended interviewer effects on the collected measurements. Such interviewer effects add uncertainty to population estimates and have the potential to lead to erroneous inferences. This study examines interviewer effects on the measurement of physical performance in a cross-national and longitudinal setting using data from the Survey of Health, Ageing and Retirement in Europe. The analyzed biomeasures exhibited moderate-to-large interviewer effects on the measurements, which varied across biomeasure types and across countries. Our findings demonstrate the necessity to better understand the origin of interviewer-related measurement errors in biomeasure collection and account for these errors in statistical analyses of biomeasure data.

Spatially varying plankton synchrony patterns at seasonal and interannual scales in a well‐connected shelf sea

AbstractSpatial population synchrony, defined as spatial covariation in population density fluctuations, exists across different temporal and spatial scales. Determining the degree of spatial synchrony is useful for inferring environmental drivers of population variability in the wake of climate change. In this study, we applied novel statistical methods to detect spatial synchrony patterns of Calanus finmarchicus on the Northeast U.S. Shelf at multiple spatiotemporal scales using unevenly distributed data. Our results reveal that C. finmarchicus subpopulations connected by advection are not necessarily in synchrony, indicating that the degree of synchrony is likely influenced by heterogeneity of local habitats. In addition, regionally synchronous environmental conditions (e.g., sea surface temperature) may not play as significant a role in influencing subregional population dynamics as was previously hypothesized. Overlooking the spatial heterogeneity of synchronous patterns at different time scales could lead to erroneous inferences of potential environmental drivers responsible for C. finmarchicus variability.

Efficacy of machine learning image classification for automated occupancy‐based monitoring

AbstractRemote cameras have become a widespread data‐collection tool for terrestrial mammals, but classifying images can be labor intensive and limit the usefulness of cameras for broad‐scale population monitoring. Machine learning algorithms for automated image classification can expedite data processing, but image misclassifications may influence inferences. Here, we used camera data for three sympatric species with disparate body sizes and life histories – black‐tailed jackrabbits (Lepus californicus), kit foxes (Vulpes macrotis), and pronghorns (Antilocapra americana) – as a model system to evaluate the influence of competing image classification approaches on estimates of occupancy and inferences about space use. We classified images with: (i) single review (manual), (ii) double review (manual by two observers), (iii) an automated‐manual review (machine learning to cull empty images and single review of remaining images), (iv) a pretrained machine‐learning algorithm that classifies images to species (base model), (v) the base model accepting only classifications with ≥95% confidence, (vi) the base model trained with regional images (trained model), and (vii) the trained model accepting only classifications with ≥95% confidence. We compared species‐specific results from alternative approaches to results from double review, which reduces the potential for misclassifications and was assumed to be the best approximation of truth. Despite high classification success, species‐level misclassification rates for the base and trained models were sufficiently high to produce erroneous occupancy estimates and inferences related to space use across species. Increasing the confidence thresholds for image classification to 95% did not consistently improve performance. Classifying images as empty (or not) offered a reasonable approach to reduce effort (by 97.7%) and facilitated a semi‐automated workflow that produced reliable estimates and inferences. Thus, camera‐based monitoring combined with machine learning algorithms for image classification could facilitate monitoring with limited manual image classification.

Unraveling preference heterogeneity in willingness-to-pay for enhanced road safety: A hybrid approach of machine learning and quantile regression

Investing in road safety enhancement programs highly depends on the economic valuation of road traffic accidents and their outcomes. Such evaluation underpins road safety interventions in cost-benefit analysis. To this end, understanding and modeling public willingness-to-pay for enhanced road safety have received significant attention in the past few decades. However, despite considerable modeling efforts, some issues still persist in earlier studies, namely, (i) using standard regression approaches that assume a homogeneous impact of explanatory variables on willingness-to-pay, not accounting for heterogeneity, and depends on a priori distribution of the dependent variable, and (ii) the absence of higher-order interactions from models, leading to omitted variable bias and erroneous model inferences. To overcome this critical research gap, our study proposes a new modeling framework, integrating a machine learning technique (decision tree) to identify a priori relationships for higher-order interactions and a quantile regression model to account for heterogeneity along the entire range of willingness-to-pay. The proposed framework examines the determinants of willingness-to-pay for enhanced road safety using a sample of car drivers from Peshawar, Pakistan. Modeling results indicate that variables not significant in a linear model become significant at specific quantiles of the willingness-to-pay distribution. Further, including higher-order interactions among the explanatory variables provides additional insights into the complex relationship between willingness-to-pay and its determinants. In addition, willingness-to-pay for fatal and severe injury risk reductions is estimated at different quartiles and used to calculate the values of corresponding risk reductions. Overall, the proposed framework provides a better understanding of public sensitivities to willingness-to-pay for enhanced road safety.

East is east … or is it? Racialization of Asian, Middle Eastern, and Pacific Islander persons.

The conventional use of racial categories in health research naturalizes "race" in problematic ways that ignore how racial categories function in service of a White-dominated racial hierarchy. In many respects, racial labels are based on geographic designations. For instance, "Asians" are from Asia. Yet, this is not always a tenable proposition. For example, Afghanistan resides in South Asia, and shares a border with China and Pakistan. Yet, people from Afghanistan are not considered Asian, but Middle Eastern, by the US Census. Furthermore, people on the west side of the Island of New Guinea are considered Asian, whereas those on the eastern side are considered Pacific Islander. In this article, we discuss the complexity of the racial labels related to people originating from Oceania and Asia, and, more specifically, those groups commonly referred to as Pacific Islander, Middle Eastern, and Asian. We begin with considerations of the aggregation fallacy. Just as the ecological fallacy refers to erroneous inferences about individuals from group data, the aggregation fallacy refers to erroneous inferences about subgroups (eg, Hmong) from group data (ie, all Asian Americans), and how these inferences can contribute to stereotypes such as the "model minority." We also examine how group averages can be influenced merely by the composition of the subgroups, and how these, in turn, can be influenced by social policies. We provide a historical overview of some of the issues facing Pacific Islander, Middle Eastern, and Asian communities, and conclude with directions for future research.

The consequences of neglected confounding and interactions in mixed-effects meta-regression: An illustrative example.

Analysts seldom include interaction terms in their meta-regression model, which can introduce bias if an interaction is present. We illustrate this by reanalysing a meta-regression study in acute heart failure. Based on a total of 285 studies, the 1-year mortality rate related to acute heart failure is considered and the connection to the study-level covariates year of recruitment and average age of study participants are of interest. We show that neglecting a possibly confounding variable and an interaction term might lead to erroneous inference and conclusions. Based on our results and accompanying simulations, we recommend to include possible confounders and interaction terms, whenever they are plausible, in mixed-effects meta-regression models.

A controlled transfer entropy approach to detect asymmetric interactions in heterogeneous systems

Transfer entropy is emerging as the statistical approach of choice to support the inference of causal interactions in complex systems from time-series of their individual units. With reference to a simple dyadic system composed of two coupled units, the successful application of net transfer entropy-based inference relies on unidirectional coupling between the units and their homogeneous dynamics. What happens when the units are bidirectionally coupled and have different dynamics? Through analytical and numerical insights, we show that net transfer entropy may lead to erroneous inference of the dominant direction of influence that stems from its dependence on the units’ individual dynamics. To control for these confounding effects, one should incorporate further knowledge about the units’ time-histories through the recent framework offered by momentary information transfer. In this realm, we demonstrate the use of two measures: controlled and fully controlled transfer entropies, which consistently yield the correct direction of dominant coupling irrespective of the sources and targets individual dynamics. Through the study of two real-world examples, we identify critical limitations with respect to the use of net transfer entropy in the inference of causal mechanisms that warrant prudence by the community.

Deep learning models challenge the prevailing assumption that face-like effects for objects of expertise support domain-general mechanisms.

The question of whether task performance is best achieved by domain-specific, or domain-general processing mechanisms is fundemental for both artificial and biological systems. This question has generated a fierce debate in the study of expert object recognition. Because humans are experts in face recognition, face-like neural and cognitive effects for objects of expertise were considered support for domain-general mechanisms. However, effects of domain, experience and level of categorization, are confounded in human studies, which may lead to erroneous inferences. To overcome these limitations, we trained deep learning algorithms on different domains (objects, faces, birds) and levels of categorization (basic, sub-ordinate, individual), matched for amount of experience. Like humans, the models generated a larger inversion effect for faces than for objects. Importantly, a face-like inversion effect was found for individual-based categorization of non-faces (birds) but only in a network specialized for that domain. Thus, contrary to prevalent assumptions, face-like effects for objects of expertise do not support domain-general mechanisms but may originate from domain-specific mechanisms. More generally, we show how deep learning algorithms can be used to dissociate factors that are inherently confounded in the natural environment of biological organisms to test hypotheses about their isolated contributions to cognition and behaviour.

An argument in support of the development of regionalized standard taxonomic efforts for benthic Macroinvertebrate-Based monitoring

Taxonomic harmonization for benthic macroinvertebrate abundance data is time consuming. If undertaken lightly it can lead to erroneous inferences regarding taxon-specific responses to environmental gradients, variation in benthic index scores, and increased noise in community comparison efforts. Standardization of taxonomic efforts and nomenclature is intended to minimize between and among lab data reporting variability, and therefore optimize the effectiveness of data analyses related to complex aquatic communities. Here we present an argument for the development of regional standard taxonomic efforts, and encourage data consumers to accept infra-rank nomenclature when it is accompanied by a well-documented and repeatable source such as a standard taxonomic effort. We present this argument from our collective experiences from within the boundaries of the United States of America but these concepts can be applied to any ecological region globally and transcend jurisdictional boundaries.

Quantitative proteomics I.: Concept, design, and planning of quantitative proteomics experiments.

Quantitative proteomics I.: Concept, design, and planning of quantitative proteomics experiments.

Hypotheses Testing from Complex Survey Data Using Bootstrap Weights: A Unified Approach

ABSTRACT Standard statistical methods without taking proper account of the complexity of a survey design can lead to erroneous inferences when applied to survey data due to unequal selection probabilities, clustering, and other design features. In particular, the Type I error rates of hypotheses tests using standard methods can be much larger than the nominal significance level. Methods incorporating design features in testing hypotheses have been proposed, including Wald tests and quasi-score tests that involve estimated covariance matrices of parameter estimates. In this article, we present a unified approach to hypothesis testing without requiring estimated covariance matrices or design effects, by constructing bootstrap approximations to quasi-likelihood ratio statistics and quasi-score statistics and establishing its asymptotic validity. The proposed method can be easily implemented without a specific software designed for complex survey sampling. We also consider hypothesis testing for categorical data and present a bootstrap procedure for testing simple goodness of fit and independence in a two-way table. In simulation studies, the Type I error rates of the proposed approach are much closer to their nominal significance level compared with the naive likelihood ratio test and quasi-score test. An application to an educational survey under a logistic regression model is also presented. Supplementary materials for this article are available online.

On the Common but Problematic Specification of Conflated Random Slopes in Multilevel Models

For multilevel models (MLMs) with fixed slopes, it has been widely recognized that a level-1 variable can have distinct between-cluster and within-cluster fixed effects, and that failing to disaggregate these effects yields a conflated, uninterpretable fixed effect. For MLMs with random slopes, however, we clarify that two different types of slope conflation can occur: that of the fixed component (termed fixed conflation) and that of the random component (termed random conflation). The latter is rarely recognized and not well understood. Here we explain that a model commonly used to disaggregate the fixed component—the contextual effect model with random slopes—troublingly still yields a conflated random component. Negative consequences of such random conflation have not been demonstrated. Here we show that they include erroneous interpretation and inferences about the substantively important extent of between-cluster differences in slopes, including either underestimating or overestimating such slope heterogeneity. Furthermore, we show that this random conflation can yield inappropriate standard errors for fixed effects. To aid researchers in practice, we delineate which types of random slope specifications yield an unconflated random component. We demonstrate the advantages of these unconflated models in terms of estimating and testing random slope variance (i.e., improved power, Type I error, and bias) and in terms of standard error estimation for fixed effects (i.e., more accurate standard errors), and make recommendations for which specifications to use for particular research purposes.

To model or not to model: false positive detection error in camera surveys

AbstractOccupancy models are commonly used with motion‐sensitive camera data to estimate patterns of species occurrence while accounting for false negative detection error (i.e., the species is present but not detected). False positive detection error (i.e., the species is not present but is detected) is present in camera data sets, especially when morphologically similar species co‐occur. Researchers use different approaches to address this problem: ignore the potential for false positive detections, remove all ambiguous detections and treat them as non‐detections, or model false positive detection error by dividing detections into ambiguous detections (could be true or false positives) and unambiguous detections (true positives). We performed a simulation study to compare these 3 strategies. To implement these modeling strategies, detections must be classified as ambiguous or unambiguous, or all ambiguous detections must be re‐classified as non‐detections. We also performed a simulation study to assess the impact of researcher confidence in the designation of ambiguous and unambiguous detections. Ignoring false positive detection error resulted in biased parameter estimates, whereas removing ambiguous detections and modeling false positive detections resulted in similar estimates of occupancy probability (ψ) in most situations. Researcher over‐confidence (i.e., the tendency for observers to overestimate their own ability) positively biased estimates of ψ. Moderate under‐confidence did not increase bias or decrease precision in estimates of ψ. Consistent with the patterns observed in simulations, analysis of example data from a chipmunk (Neotamias minimus atristriatus) population in the Sacramento Mountains of south‐central New Mexico during 2019 indicated that removing ambiguous detections and modeling false positives resulted in similar estimates of ψ and that over‐confidence biased estimates of ψ. Our results expand on previous literature, suggesting that removing ambiguous detections provides similar estimates of occupancy compared to modeling false positives in many scenarios, and emphasizing the importance of the designation of ambiguous and unambiguous detections. We provide guidance on simple methods to define ambiguous and unambiguous detections, thus mitigating the chances for erroneous inferences.

A cognitive process modeling framework for the ABCD study stop-signal task

The Adolescent Brain Cognitive Development (ABCD) Study is a longitudinal neuroimaging study of unprecedented scale that is in the process of following over 11,000 youth from middle childhood though age 20. However, a design feature of the study's stop-signal task violates "context independence", an assumption critical to current non-parametric methods for estimating stop-signal reaction time (SSRT), a key measure of inhibitory ability in the study. This has led some experts to call for the task to be changed and for previously collected data to be used with caution. We present a cognitive process modeling framework, the RDEX-ABCD model, that provides a parsimonious explanation for the impact of this design feature on “go” stimulus processing and successfully accounts for key behavioral trends in the ABCD data. Simulation studies using this model suggest that failing to account for the context independence violations in the ABCD design can lead to erroneous inferences in several realistic scenarios. However, we demonstrate that RDEX-ABCD effectively addresses these violations and can be used to accurately measure SSRT along with an array of additional mechanistic parameters of interest (e.g., attention to the stop signal, cognitive efficiency), advancing investigators’ ability to draw valid and nuanced inferences from ABCD data. Availability of data and materialsData from the ABCD Study are available through the NIH Data Archive (NDA): nda.nih.gov/abcd. Code for all analyses featured in this study is openly available on the Open Science Framework (OSF): osf.io/2h8a7/.

SAMPLING METHODS AND APPROPRIATE SAMPLE SIZE DETERMINATION: A CONCISE OVERVIEW

Sample size determination has been a controversial issue for many applied researches, such that incorrect determination of how large a sample should be may lead to erroneous inferences in the study. For this reason, in this paper it has been aimed to highlight the general criteria and formulations for detecting the required minimum sample size especially in probability sampling methods -which can be summarized as simple random sampling, systematic sampling, stratified sampling and cluster sampling- and various research designs. Besides, the place of statistical power and effect size on sample size determination and the sample size computations for regression analysis based on the study by Cohen (1988) have also been mentioned.

A unified Gaussian copula methodology for spatial regression analysis

Spatially referenced data arise in many fields, including imaging, ecology, public health, and marketing. Although principled smoothing or interpolation is paramount for many practitioners, regression, too, can be an important (or even the only or most important) goal of a spatial analysis. When doing spatial regression it is crucial to accommodate spatial variation in the response variable that cannot be explained by the spatially patterned explanatory variables included in the model. Failure to model both sources of spatial dependence—regression and extra-regression, if you will—can lead to erroneous inference for the regression coefficients. In this article I highlight an under-appreciated spatial regression model, namely, the spatial Gaussian copula regression model (SGCRM), and describe said model’s advantages. Then I develop an intuitive, unified, and computationally efficient approach to inference for the SGCRM. I demonstrate the efficacy of the proposed methodology by way of an extensive simulation study along with analyses of a well-known dataset from disease mapping.

Matrix completion under complex survey sampling.

Multivariate nonresponse is often encountered in complex survey sampling, and simply ignoring it leads to erroneous inference. In this paper, we propose a new matrix completion method for complex survey sampling. Different from existing works either conducting row-wise or column-wise imputation, the data matrix is treated as a whole which allows for exploiting both row and column patterns simultaneously. A column-space-decomposition model is adopted incorporating a low-rank structured matrix for the finite population with easy-to-obtain demographic information as covariates. Besides, we propose a computationally efficient projection strategy to identify the model parameters under complex survey sampling. Then, an augmented inverse probability weighting estimator is used to estimate the parameter of interest, and the corresponding asymptotic upper bound of the estimation error is derived. Simulation studies show that the proposed estimator has a smaller mean squared error than other competitors, and the corresponding variance estimator performs well. The proposed method is applied to assess the health status of the U.S. population.

Artificial intelligence and dermatology.

The approach of artificial intelligence (AI) systems can especially support doctors who base their work on the interpretation of digital images. Despite undoubted favorable results, the validation of diagnostic algorithms in clinical practice has not yet been demonstrated. In the dermatological field, a study found the poor performance of AI systems in diagnosing skin lesions that better reflect clinical practice, that is characterized by the presence of "artifacts" such as crusts or ulcerations above the lesion, hair or pen marks, pigmentation. The algorithms also correctly classified the category of images not present in the training dataset in only 11% of cases. In almost half of the cases the images were assigned to the category of neoplasms. In clinical reality, such false positivity predictions would lead to an increase in unnecessary biopsies with a related burden of worry and anxiety for patients and their families. This confirms that the value of the data is not in their breadth but in the validity of the path that led to their use, in order not to arrive at erroneous causal inferences that could lead to poor generalizability of the results and potentially serious diagnostic errors.