Invalid Inferences Research Articles

Modeling Skipped Items in Cognitive Diagnostic Assessments

In cognitive diagnosis assessments, examinees frequently skip some items due to various reasons, such as time constraints, lack of confidence, or the perception of item difficulty. These skipped items are usually due to lacking specific cognitive attributes or knowledge structures. However, many research studies conventionally rely on complete-case analysis, potentially compromising the validity of results by ignoring skipped items. This omission can lead to invalid inferences about the attributes profiles of examinees and introduce bias into item parameters. This study aims to develop statistical models for effectively handling missing data resulting from skipped items. Specifically, we employ an item response theory model for missing indicators and utilize the deterministic inputs, noisy “and” gate (DINA) model to describe cognitive item responses. Furthermore, we introduce a higher-order structure to describe the correlation between the higher-order ability parameters and skipping propensity parameters and the correlation of the item parameters. The proposed new model fills the gaps in missing data handling, providing a more precise evaluation for the examinees who skip items and yielding more accurate research results. This enhances our understanding of cognitive processes and strengthens support for educational policies, aligning them better with student needs and optimizing decision-making.

Covariate-adjusted generalized pairwise comparisons in small samples.

Semiparametric probabilistic index models allow for the comparison of two groups of observations, whilst adjusting for covariates, thereby fitting nicely within the framework of generalized pairwise comparisons (GPC). As with most regression approaches in this setting, the limited amount of data results in invalid inference as the asymptotic normality assumption is not met. In addition, separation issues might arise when considering small samples. In this article, we show that the parameters of the probabilistic index model can be estimated using generalized estimating equations, for which adjustments exist that lead to estimators of the sandwich variance-covariance matrix with improved finite sample properties and that can deal with bias due to separation. In this way, appropriate inference can be performed as is shown through extensive simulation studies. The known relationships between the probabilistic index and other GPC statistics allow to also provide valid inference for example, the net treatment benefit or the success odds.

Explicit inference: A meta-replication of SEER cancer registry research evaluating the Affordable Care Act's Medicaid expansion.

Among the provisions within the Affordable Care Act (ACA), expanding Medicaid was arguably the greatest contributor to increasing access to care. For over a decade, researchers have investigated how Medicaid expansion impacted cancer outcomes. Over this same decade, statistical theory illuminated how state-based policy research could be compromised by invalid inference. After reviewing the literature to identify the inference strategies of state-based cancer registry Medicaid expansion research, this study aimed to assess how inference decisions could change the interpretation of Medicaid expansion's impact on staging, treatment, and mortality in cancer patients. Cancer case data (2000-2019) was obtained from the Surveillance, Epidemiology, End Results (SEER) programme. Cases included all cancer sites combined, top 10 cancer sites combined, and three screening amenable cancers (colorectal, female breast, female cervical). A Difference-in-Differences design estimated the association between Medicaid expansion and four binary outcomes: distant stage, initiating treatment >1 month after diagnosis, no surgery recommendation, and death. Three inference techniques were compared: (1) traditional, (2) cluster, and (3) Wild Cluster Bootstrap. Data was accessed via SEER*Stat. Estimating standard errors via traditional inference would suggest that Medicaid expansion was associated with delayed treatment initiation and surgery recommendations. Traditional and clustered inference also suggested that Medicaid expansion reduced mortality. Inference using Wild Cluster Bootstrap techniques never rejected the null hypotheses. This study reiterates the importance of explicit inference. Future state-based, cancer policy research can be improved by incorporating emerging techniques. These findings warrant caution when interpreting prior SEER research reporting significant effects of Medicaid expansion on cancer outcomes, especially studies that did not explicitly define their inference strategy.

Identifying Clusters of Less-Skilled College Student Readers Based on Cognitive Processes

ABSTRACT The purpose of this study was to examine clusters of less-skilled college readers. College students with below average reading comprehension skills (N = 77) read and thought aloud about four texts, recalled the texts, and completed standardized assessments of reading skills. Based on the findings of cluster analyses of the cognitive processes relevant to creating coherent mental representations of the texts, two clusters of readers were noted. One cluster had more paraphrases and inaccurate or irrelevant connections to background knowledge (invalid inferences) as well as fewer recalled ideas from the texts than the second cluster, which had more accurate and relevant connections to background knowledge (valid inferences). The findings inform developing and personalizing reading interventions for college students.

Analysis of Differentially Private Synthetic Data: A Measurement Error Approach

Differentially private (DP) synthetic datasets have been receiving significant attention from academia, industry, and government. However, little is known about how to perform statistical inference using DP synthetic datasets. Naive approaches that do not take into account the induced uncertainty due to the DP mechanism will result in biased estimators and invalid inferences. In this paper, we present a class of maximum likelihood estimator (MLE)-based easy-to-implement bias-corrected DP estimators with valid asymptotic confidence intervals (CI) for parameters in regression settings, by establishing the connection between additive DP mechanisms and measurement error models. Our simulation shows that our estimator has comparable performance to the widely used sufficient statistic perturbation (SSP) algorithm in some scenarios but with the advantage of releasing a synthetic dataset and obtaining statistically valid asymptotic CIs, which can achieve better coverage when compared to the naive CIs obtained by ignoring the DP mechanism.

Where, When, and Why Is Zeno’s Arrow Unmoved? – A Note on the Zenonian Challenge in Aristotle’s Physics, Book VI

Abstract Zeno’s arrow does not move “in the now” (Phys. VI 8, 239b2) or, equivalently, “in the place it is” (DK 29 B 4). Zeno concludes from this that the arrow does not move at all. In Aristotle (ibid. 9, 239b5–9, 31–33), Zeno’s argument takes the form of an invalid inference from instants to periods of time. Insofar as it fails to bring out an inconsistency in Aristotle’s account of motion, the paradox is thus eliminated. That instantaneous motion is a contradiction in terms belongs to the common ground Aristotle shares with Zeno. That’s why information concerning Zeno’s relevant argument must be sought elsewhere (see section 3). Aristotle’s discussion of the Arrow pertains to the conclusions drawn from the denial of instantaneous motion. But the argument he refutes is of his own making, or so I will argue (section 2). In particular, Aristotle fails to address the presentist commitment adumbrated in his account, which would leave nothing beyond the denial of instantaneous motion for Zeno to prove. Presentism is no option in Phys. VI where a tenseless account of time and motion is offered. I will argue that Aristotle’s account of the tenses in De interpretatione is prone to Zeno’s argument if the present is conceived as an instant – which it arguably is not (section 4).

The Failure of Roy Bhaskar's Explanatory Critique

AbstractAccording to Roy Bhaskar, social science can derive values from social facts by a process called “explanatory critique.” Bhaskar offers two different versions of explanatory critique: a belief‐based version and a need‐based version. Both versions are faced with a difficult objection. They seem either to employ an invalid inference or to assume the values that they are attempting to derive. I argue that at least the need‐based version of Bhaskar's explanatory critique falls to the objection, and that the belief‐based version on its own is insufficient. Bhaskar anticipates the objection and offers a defense. I show that his defense is unsuccessful. I also suggest a Baskar‐inspired alternative explanatory critique.

Trade efficiency under FTA for Thailand’s agricultural exports: copula-based gravity stochastic frontier model

This study investigated the trade efficiency and trade effects under Free Trade Agreements for Thailand’s agricultural exports. There are five main trading partners comprising China, Japan, Australia, New Zealand, and India. The important agricultural commodities of rubber, cassava, fruits, vegetables, and herbs were assessed from 1998 to 2019. In a traditional stochastic frontier model the two error components of symmetric noise ({nu }_{j}) and a non-negative inefficiency ({u}_{j}) are assumed to be independent. This may result in invalid inferences due to misspecification. To address this obstacle, copula-based Gravity Stochastic Frontier Models (GSFM) using panel criteria were constructed to estimate trade efficiency. Empirically, the Student-t copula-based SFM minimizes both AIC and BIC. According to their mean TE, China (0.48) and Japan (0.48) had the highest export efficiencies followed by India (0.41), New Zealand (0.39), and Australia (0.33) in rank order. Hence, Thailand should pursue more FTA negotiations with the trading partners. Moreover, they should promote miscellaneous behind-the-border barriers to stimulate flows of goods to enhance the country’s trade efficiency substantially.

Measurement invariance of maternal depressive symptoms across the first 2 years since birth and across racial group, education, income, primiparity, and age.

Up to 19% of postpartum mothers experience depressive symptoms, which are associated with infant development. Thus, research examining postpartum depression has implications for mothers' and infants' well-being. However, this research relies on the often-untested assumption of measurement invariance-that measures capture the same construct across time and sociodemographic characteristics. In the absence of invariance, measurement bias may confound differences across time and group, contributing to invalid inferences. In a sociodemographically diverse (40.7% African American, 58.9% White; 67.9% below two times the federal poverty line; 19.4% with less than high school education), rural, longitudinal sample (N = 1,275) of mothers, we used moderated nonlinear factor analysis (MNLFA) to examine measurement invariance of the Brief Symptom Inventory-18 (BSI-18) Depressive Symptoms subscale across time since birth, racial group, education, income, primiparity, and maternal age at childbirth. We identified evidence of differential item functioning (DIF; i.e., measurement noninvariance) as a function of racial group and education. Subsequent analyses indicated, however, that the DIF-induced bias had minimal impacts on substantive comparisons examining change over time since birth and group differences. Thus, the presence of measurement noninvariance does not appear to bias substantive comparisons using the BSI-18 Depressive Symptoms subscale across the first 2 years since birth in a sample comprising primarily African American and White mothers living in predominately rural, low-income communities. This study demonstrates the importance of assessing measurement invariance and highlights MNLFA for evaluating the impact of noninvariance as a preliminary step that increases confidence in the validity of substantive inferences. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

Electrophysiological connectivity of logical deduction: Early cortical MEG study

Complex human reasoning involves minimal abilities to extract conclusions implied in the available information. These abilities are considered “deductive” because they exemplify certain abstract relations among propositions or probabilities called deductive arguments. However, the electrophysiological dynamics which supports such complex cognitive processes has not been addressed yet. In this work we consider typically deductive logico-probabilistically valid inferences and aim to verify or refute their electrophysiological functional connectivity differences from invalid inferences with the same content (same relational variables, same stimuli, same relevant and salient features). We recorded the brain electrophysiological activity of 20 participants (age = 20.35 ± 3.23) by means of an MEG system during two consecutive reasoning tasks: a search task (invalid condition) without any specific deductive rules to follow, and a logically valid deductive task (valid condition) with explicit deductive rules as instructions. We calculated the functional connectivity (FC) for each condition and conducted a seed-based analysis in a set of cortical regions of interest. Finally, we used a cluster-based permutation test to compare the differences between logically valid and invalid conditions in terms of FC. As a first novel result we found higher FC for valid condition in beta band between regions of interest and left prefrontal, temporal, parietal, and cingulate structures. FC analysis allows a second novel result which is the definition of a propositional network with operculo-cingular, parietal and medial nodes, specifically including disputed medial deductive “core” areas. The experiment discloses measurable cortical processes which do not depend on content but on truth-functional propositional operators. These experimental novelties may contribute to understand the cortical bases of deductive processes.

Electrical analysis of logical complexity: an exploratory eeg study of logically valid/invalid deducive inference

IntroductionLogically valid deductive arguments are clear examples of abstract recursive computational procedures on propositions or on probabilities. However, it is not known if the cortical time-consuming inferential processes in which logical arguments are eventually realized in the brain are in fact physically different from other kinds of inferential processes.MethodsIn order to determine whether an electrical EEG discernible pattern of logical deduction exists or not, a new experimental paradigm is proposed contrasting logically valid and invalid inferences with exactly the same content (same premises and same relational variables) and distinct logical complexity (propositional truth-functional operators). Electroencephalographic signals from 19 subjects (24.2 ± 3.3 years) were acquired in a two-condition paradigm (100 trials for each condition). After the initial general analysis, a trial-by-trial approach in beta-2 band allowed to uncover not only evoked but also phase asynchronous activity between trials.Resultsshowed that (i) deductive inferences with the same content evoked the same response pattern in logically valid and invalid conditions, (ii) mean response time in logically valid inferences is 61.54% higher, (iii) logically valid inferences are subjected to an early (400 ms) and a late reprocessing (600 ms) verified by two distinct beta-2 activations (p-value < 0,01, Wilcoxon signed rank test).ConclusionWe found evidence of a subtle but measurable electrical trait of logical validity. Results put forward the hypothesis that some logically valid deductions are recursive or computational cortical events.

Some Modified Ridge Estimators for Handling the Multicollinearity Problem

Regression analysis is a statistical process that utilizes two or more predictor variables to predict a response variable. When the predictors included in the regression model are strongly correlated with each other, the problem of multicollinearity arises in the model. Due to this problem, the model variance increases significantly, leading to inconsistent ordinary least-squares estimators that may lead to invalid inferences. There are numerous existing strategies used to solve the multicollinearity issue, and one of the most used methods is ridge regression. The aim of this work is to develop novel estimators for the ridge parameter “γ” and compare them with existing estimators via extensive Monte Carlo simulation and real data sets based on the mean squared error criterion. The study findings indicate that the proposed estimators outperform the existing estimators.

Liberal-democratic values and philosophers' beliefs about moral expertise.

In recent decades, the discipline of bioethics has grown rapidly, as has the practice of ethical consultation. Interestingly, this new recognition of the relevance of moral philosophy to our daily life has been accompanied by skepticism among philosophers regarding the existence of moral expertise or the benefits of philosophical training. In his recent article in Bioethics, William R. Smith suggested that this skepticism is rooted in philosophers' belief that moral expertise is inconsistent with liberal-democratic values, when in fact they are compatible. In this paper, we provide a unique opportunity to empirically examine Smith's observation by utilizing and extending global data on philosophers' beliefs about moral expertise, involving 4087 philosophers from 96 countries. Our findings support Smith's theoretical observation and show that societal levels of support for liberal-democratic values are associated with greater skepticism about moral expertise. We suggest that these findings might be explained by the cognitive process of motivated reasoning and an invalid inference of "is" from "ought." Consequently, the potential tension between moral expertise and liberal-democratic values is invalidly used for rejecting the existence of moral expertise, while its main and valid implication is for how moral expertise should be applied in liberal-democratic settings.

Reciprocal causation mixture model for robust Mendelian randomization analysis using genome-scale summary data

Mendelian randomization using GWAS summary statistics has become a popular method to infer causal relationships across complex diseases. However, the widespread pleiotropy observed in GWAS has made the selection of valid instrumental variables problematic, leading to possible violations of Mendelian randomization assumptions and thus potentially invalid inferences concerning causation. Furthermore, current MR methods can examine causation in only one direction, so that two separate analyses are required for bi-directional analysis. In this study, we propose a ststistical framework, MRCI (Mixture model Reciprocal Causation Inference), to estimate reciprocal causation between two phenotypes simultaneously using the genome-scale summary statistics of the two phenotypes and reference linkage disequilibrium information. Simulation studies, including strong correlated pleiotropy, showed that MRCI obtained nearly unbiased estimates of causation in both directions, and correct Type I error rates under the null hypothesis. In applications to real GWAS data, MRCI detected significant bi-directional and uni-directional causal influences between common diseases and putative risk factors.

Support Interval for Two-Sample Summary Data-Based Mendelian Randomization.

The summary-data-based Mendelian randomization (SMR) method is gaining popularity in estimating the causal effect of an exposure on an outcome. In practice, the instrument SNP is often selected from the genome-wide association study (GWAS) on the exposure but no correction is made for such selection in downstream analysis, leading to a biased estimate of the effect size and invalid inference. We address this issue by using the likelihood derived from the sampling distribution of the estimated SNP effects in the exposure GWAS and the outcome GWAS. This likelihood takes into account how the instrument SNPs are selected. Since the effective sample size is 1, the asymptotic theory does not apply. We use a support for a profile likelihood as an interval estimate of the causal effect. Simulation studies indicate that this support has robust coverage while the confidence interval implied by the SMR method has lower-than-nominal coverage. Furthermore, the variance of the two-stage least squares estimate of the causal effect is shown to be the same as the variance used for SMR for one-sample data when there is no selection.

Weak Signal Identification and Inference in Penalized Likelihood Models for Categorical Responses

Penalized likelihood models are widely used to simultaneously select variables and estimate model parameters. However, the existence of weak signals can lead to inaccurate variable selection, biased parameter estimation, and invalid inference. Thus, identifying weak signals accurately and making valid inferences are crucial in penalized likelihood models. We develop a unified approach to identify weak signals and make inferences in penalized likelihood models, including the special case when the responses are categorical. To identify weak signals, we use the estimated selection probability of each covariate as a measure of the signal strength and formulate a signal identification criterion. To construct confidence intervals, we propose a two-step inference procedure. Extensive simulation studies show that the proposed procedure outperforms several existing methods. We illustrate the proposed method by applying it to the Practice Fusion diabetes data set.

Inferential reasoning ability moderates the influence of mediated exemplars on risk perception

ABSTRACT Research has shown that vivid exemplars bias risk perceptions, raising concerns about the role of media in shaping public opinion. However, receiver characteristics that moderate the influence of vivid exemplars remain understudied. We propose that insofar as exemplars in media portrayals represent (at times biased) samples of the population of events, inferential reasoning ability (IRA) – the knowledge of and skill at applying inferential and statistical principles – may enable people to avoid drawing invalid inferences from exemplars. In two message-exposure experiments employing different designs and stimulus materials, we find that exemplars conditionally influence perceptions of risk, depending on the receiver’s IRA. Specifically, higher IRA was associated with reduced influence of vivid exemplars. This work highlights the importance of accounting for heterogeneous audiences in risk messaging and offers tools for measuring audience IRA.

Quantifying the robustness of causal inferences: Sensitivity analysis for pragmatic social science

Social scientists seeking to inform policy or public action must carefully consider how to identify effects and express inferences because actions based on invalid inferences may not yield the intended results. Recognizing the complexities and uncertainties of social science, we seek to inform inevitable debates about causal inferences by quantifying the conditions necessary to change an inference. Specifically, we review existing sensitivity analyses within the omitted variables and potential outcomes frameworks. We then present the Impact Threshold for a Confounding Variable (ITCV) based on omitted variables in the linear model and the Robustness of Inference to Replacement (RIR) based on the potential outcomes framework. We extend each approach to include benchmarks and to fully account for sampling variability represented by standard errors as well as bias. We exhort social scientists wishing to inform policy and practice to quantify the robustness of their inferences after utilizing the best available data and methods to draw an initial causal inference.

Marginal semiparametric transformation models for clustered multivariate competing risks data.

Multivariate survival models are often used in studying multiple outcomes for right-censored data. However, the outcomes of interest often have competing risks, where standard multivariate survival models may lead to invalid inferences. For example, patients who had stem cell transplantation may experience multiple types of infections after transplant while reconstituting their immune system, where death without experiencing infections is a competing risk for infections. Such competing risks data often suffer from cluster effects due to a matched pair design or correlation within study centers. The cumulative incidence function (CIF) is widely used to summarize competing risks outcomes. Thus, it is often of interest to study direct covariate effects on the CIF. Most literature on clustered competing risks data analyses is limited to the univariate proportional subdistribution hazards model with inverse probability censoring weighting which requires correctly specifying the censoring distribution. We propose a marginal semiparametric transformation model for multivariate competing risks outcomes. The proposed model does not require modeling the censoring distribution, accommodates nonproportional subdistribution hazards structure, and provides a platform for joint inference of all causes and outcomes.

Testing latent class of subjects with structural zeros in negative binomial models with applications to gut microbiome data.

Human microbiome research has become a hot-spot in health and medical research in the past decade due to the rapid development of modern high-throughput. Typical data in a microbiome study consisting of the operational taxonomic unit counts may have over-dispersion and/or structural zero issues. In such cases, negative binomial models can be applied to address the over-dispersion issue, while zero-inflated negative binomial models can be applied to address both issues. In practice, it is essential to know if there is zero-inflation in the data before applying negative binomial or zero-inflated negative binomial models because zero-inflated negative binomial models may be unnecessarily complex and difficult to interpret, or may even suffer from convergence issues if there is no zero-inflation in the data. On the other hand, negative binomial models may yield invalid inferences if the data does exhibit excessive zeros. In this paper, we develop a new test for detecting zero-inflation resulting from a latent class of subjects with structural zeros in a negative binomial regression model by directly comparing the amount of observed zeros with what would be expected under the negative binomial regression model. A closed form of the test statistic as well as its asymptotic properties are derived based on estimating equations. Intensive simulation studies are conducted to investigate the performance of the new test and compare it with the classical Wald, likelihood ratio, and score tests. The tests are also applied to human gut microbiome data to test latent class in microbial genera.