Statistical Misconceptions Research Articles

Statistical Misconceptions, Awareness, and Attitudes towards Open Science Practices in Slovak Psychology Researchers

Statistical Misconceptions, Awareness, and Attitudes towards Open Science Practices in Slovak Psychology Researchers

Using Clickers to Address Common Errors and Misconceptions in Statistics

ABSTRACT Using clickers in the statistics classroom can help students identify and understand common errors and misconceptions through a combination of surprise and discussion. Students are presented with multiple-choice questions that they discuss with each other and then vote on; a class-wide discussion follows. Questions for which many students vote for the same wrong answer and questions that reveal a common misconception jolt students’ attention and lead to fruitful discussions. To identify the questions that most effectively target misconceptions, we recorded the percentage of the class voting on each of the possible responses on every clicker question used in 28 sections of introductory statistics courses taught by 3 instructors at 2 institutions. For each question, we identified the distractor that attracted the highest percent of votes on average. We present and discuss the six questions for which this average was at least 50%. We find that each of these question statements is fairly brief and tightly focused on a particular common error or misconception.

The Multidimensional Indicator of Extractives-based Development (MINDEX): A new approach to measuring resource wealth and dependence

• There are considerable limitations to existing natural resource measurements. • Inaccurate measurements have biased studies on the impact of natural resources on development. • Natural resource wealth is multidimensional and come in several shapes. • The MINDEX represents a rethinking of natural resource measurements. • It also provides a diagnostic and policy evaluation tool for the extractives sector. Despite the vast amount of academic work estimating the impact of natural resources on development, very little attention has been devoted to the implications of using one type of natural resource measurement over another. This study fills this important gap in two ways. Firstly, it puts forward the biases and statistical misconceptions associated with different measurements of resource wealth, which have often led to the wrong classification of resource-poor countries as resource-rich and vice versa. As a result of the limitations of existing measurements, the discourse around extractives-based development has tended to lump various countries together, considering them all to be ‘resource-rich’, which is misleading. Instead, this paper shows that resource wealth and dependence are multifaceted. Secondly, in contrast to the conventional measurements that have relied on different indicators of resource wealth in isolation from one another, this study sheds light on the need for a multidimensional approach to measuring resource endowment. I propose a new indicator, the MINDEX, which weights six different variables of both resource abundance and dependence across several dimensions (extractives reserves, production, exports, and government revenues) that relate to the different steps of resource exploitation chain to harness natural resources for development. Because of its methodology, the MINDEX can also serve as a diagnostic tool that contributes to identifying some of the extractives-related policy challenges that a given country may face at a given time (such as illegal commodity smuggling, poor appropriation/taxation of commodity revenues, limited production capacity of existing deposits, vulnerability to commodity price fluctuations, and acute commodity dependence). It therefore also responds to the need for a new measure of extractives-based development to indicate whether a country is moving in the right or wrong direction over time and has clear relevance for informing resource mobilization dynamics and development strategies.

It’s time for sex in cognitive neuroscience

ABSTRACT In a discussion paper published in the special issue of Cognitive Neuroscience, Sex Differences in the Brain, we investigated whether certain experimental parameters contributed to findings in functional magnetic resonance imaging studies of sex differences during long-term memory. Experimental parameters included: the number of participants, stimulus type(s), whether or not performance was matched, whether or not sex differences were reported, the type of between-subject statistical test used, and the contrast(s) employed. None of these parameters determined whether or not differences were observed, as all included studies reported sex differences. We also conducted a meta-analysis to determine if there were any brain regions consistently activated to a greater degree in either sex. The meta-analysis identified sex differences (male > female) in the lateral prefrontal cortex, visual processing regions, parahippocampal cortex, and the cerebellum. We received eight commentaries in response to that paper. Commentaries called for an expanded discussion on various topics including the influence of sex hormones, the role of gender (and other social factors), the pros and cons of equating behavioral performance between the sexes, and interpreting group differences in patterns of brain activity. There were some common statistical assumptions discussed in the commentaries regarding the ‘file drawer’ issue (i.e., the lack of reporting of null results) and effect size. The current paper provides further discussion of the various topics brought up in the commentaries and addresses some statistical misconceptions in the field. Overall, the commentaries echoed a resounding call to include sex as a factor in cognitive neuroscience studies.

Systematic review of education and practical guidance on regression modeling for medical researchers who lack a strong statistical background: Study protocol.

In the last decades, statistical methodology has developed rapidly, in particular in the field of regression modeling. Multivariable regression models are applied in almost all medical research projects. Therefore, the potential impact of statistical misconceptions within this field can be enormous Indeed, the current theoretical statistical knowledge is not always adequately transferred to the current practice in medical statistics. Some medical journals have identified this problem and published isolated statistical articles and even whole series thereof. In this systematic review, we aim to assess the current level of education on regression modeling that is provided to medical researchers via series of statistical articles published in medical journals. The present manuscript is a protocol for a systematic review that aims to assess which aspects of regression modeling are covered by statistical series published in medical journals that intend to train and guide applied medical researchers with limited statistical knowledge. Statistical paper series cannot easily be summarized and identified by common keywords in an electronic search engine like Scopus. We therefore identified series by a systematic request to statistical experts who are part or related to the STRATOS Initiative (STRengthening Analytical Thinking for Observational Studies). Within each identified article, two raters will independently check the content of the articles with respect to a predefined list of key aspects related to regression modeling. The content analysis of the topic-relevant articles will be performed using a predefined report form to assess the content as objectively as possible. Any disputes will be resolved by a third reviewer. Summary analyses will identify potential methodological gaps and misconceptions that may have an important impact on the quality of analyses in medical research. This review will thus provide a basis for future guidance papers and tutorials in the field of regression modeling which will enable medical researchers 1) to interpret publications in a correct way, 2) to perform basic statistical analyses in a correct way and 3) to identify situations when the help of a statistical expert is required.

Assessing Nigerian Secondary School Students’ Misconceptions in Five Basic Statistical Concepts

The study was designed to assess and document students’ right conceptions and misconceptions of five basic concepts in statistics. It employed the survey design to randomly select 4332 participants from 114 Senior Secondary Schools across Edo State, Nigeria. The instrument was a two-tier multiple choice diagnostic test, which was validated and with a K-R Formular 20 reliability coefficient of 0.85. Three research questions were raised to guide the study. Data were analyzed using frequency counts and percentages. Results showed low students’ right conceptions of basic statistical concepts and with majority of students having some nine misconceptions. It was recommended that teachers and textbook writers should use conceptual change instructional strategies to identify and change students’ misconceptions in statistics in order to improve students’ understanding and achievement in mathematics.

Refutation texts compensate for detrimental effects of misconceptions on comprehension and metacomprehension accuracy and support transfer.

Misconceptions impair not only learners’ comprehension of a text but also the accuracy with which they judge their comprehension, that is, metacomprehension accuracy. Refutation texts are beneficial to elicit conceptual-change processes and thus to overcome the detrimental impact of misconceptions on comprehension. However, it is unclear whether refutation texts and the conceptual-change processes induced by reading them benefit metacomprehension accuracy in addition to comprehension and how robust these effects are when reading further texts. We conducted an experiment in which we asked 80 student teachers with a varying number of statistical misconceptions to read either a standard or a refutation text on statistics and assessed their comprehension and metacomprehension. In a subsample, we examined the students’ conceptual-change processes during reading with the think-aloud methodology. Subsequently, we asked the students to read a research report that required the application of the previously acquired statistical knowledge and assessed their comprehension and metacomprehension again. The results revealed that the refutation text compensated for the detrimental impact of misconceptions on comprehension and metacomprehension accuracy. The think-aloud data indicated that when reading the refutation text, students with more misconceptions engaged in conceptual-change processes more actively, which in turn supported their comprehension and metacomprehension accuracy. Compensatory effects of the refutation text were also found when the students read the research report. In sum, the findings show that refutation texts are beneficial for both comprehension and metacomprehension accuracy and that these benefits transfer to situations where the acquired knowledge has to be put into practice. (PsycINFO Database Record (c) 2019 APA, all rights reserved)

Essential Statistical Concepts for Research in Speech, Language, and Hearing Sciences.

Purpose Clinicians depend on the accuracy of research in the speech, language, and hearing sciences to improve assessment and treatment of patients with communication disorders. Although this work has contributed to great advances in clinical care, common statistical misconceptions remain, which deserve closer inspection in the field. Challenges in applying and interpreting traditional statistical methods with behavioral data from humans have led to difficulties with replication and reproducibility in other allied scientific fields, including psychology and medicine. The importance of research in our fields of study for advancing science and clinical care for our patients means that the choices of statistical methods can have far-reaching, real-world implications. Method The goal of this article is to provide an overview of fundamental statistical concepts and methods that are used in the speech, language, and hearing sciences. Results We reintroduce basic statistical terms such as the p value and effect size, as well as recommended procedures for model selection and multiple comparisons. Conclusions Research in the speech, language, and hearing sciences can have a profound positive impact on the lives of individuals with communication disorders, but the validity of scientific findings in our fields is enhanced when data are analyzed using sound statistical methods. Misunderstanding or misinterpretation of basic statistical principles may erode public trust in research findings. Recommendations for practices that can help minimize the likelihood of errors in statistical inference are provided. Supplemental Material https://doi.org/10.23641/asha.7849223.

Pre-service teacher statistical misconceptions during teacher preparation program

Pre-service teacher statistical misconceptions during teacher preparation program

Multilevel modeling myths.

The use of multilevel modeling (MLM) to analyze nested data has grown in popularity over the years in the study of school psychology. However, with the increase in use, several statistical misconceptions about the technique have also proliferated. We discuss some commonly cited myths and golden rules related to the use of MLM, explain their origin, and suggest approaches to dealing with certain issues. Misunderstandings related to the use of the intraclass correlation, design effects, minimum sample size, multilevel factor structures, model R², and the misestimation of standard errors are reviewed. Many of the cited myths have much truth in them-though at times, researchers may not be aware of the exceptions to the rules that prevent their overall generalization. Although nesting should be accounted for, researchers should realize that MLM, which is a powerful and flexible technique, is not the only method that can be used to account for the clustering effect. (PsycINFO Database Record

The double curse of misconceptions: misconceptions impair not only text comprehension but also metacomprehension in the domain of statistics

Research shows that misconceptions are usually detrimental to text comprehension. However, whether misconceptions also impair metacomprehension accuracy, that is, the accuracy with which one self-assesses one’s text comprehension, has received far less attention. We conducted a study in which we examined students’ (N = 47) comprehension and metacomprehension accuracy (prediction accuracy and postdiction accuracy) of a statistics text as a function of their statistical misconceptions. Text comprehension and metacomprehension accuracy referred to both conceptual and procedural aspects of statistics. The results showed that students who had more misconceptions achieved poorer conceptual text comprehension and, at the same time, provided more overconfident predictions of their conceptual and procedural text comprehension than students who had fewer misconceptions. In contrast, postdiction accuracy of conceptual and procedural text comprehension was not affected by misconceptions.

PERSONAL EPISTEMOLOGIES OF STATISTICIANS IN ACADEMIA: AN EXPLORATORY STUDY

In this exploratory study, we investigated the personal epistemologies of statisticians in academia with the aim of offering some insight into what might be an availing epistemology for learning statistics. Findings from in-depth, semi-structured interviews with six academics in the UK currently researching within the field of statistics showed that their statistical personal epistemologies were consistent with constructivist theories of learning. Based on these findings, we proposed an integrated model to represent participants’ beliefs about knowledge and knowing in statistics. Furthermore, we found differences between participants’ mathematical and statistical personal epistemologies, and we offered an argument for domain-specificity of personal epistemologies. The implications of our findings for research in statistics education and statistical misconceptions are discussed. First published November 2017 at Statistics Education Research Journal Archives

Personal Epistemologies of Statisticians in Academia: An Exploratory Study.

In this exploratory study, we investigated the personal epistemologies of statisticians in academia with the aim of offering some insight into what might be an availing epistemology for learning statistics. Findings from in-depth, semi-structured interviews with six academics in the UK currently researching within the field of statistics showed that their statistical personal epistemologies were consistent with constructivist theories of learning. Based on these findings, we proposed an integrated model to represent participants’ beliefs about knowledge and knowing in statistics. Furthermore, we found differences between participants’ mathematical and statistical personal epistemologies, and we offered an argument for domain-specificity of personal epistemologies. The implications of our findings for research in statistics education and statistical misconceptions are discussed. First published November 2017 at Statistics Education Research Journal Archives

Advancing road ecology in Africa with robust analyses and cautious inferences: a response to Jacksonet al. (2017)

Jackson et al. (2017) have written an extensive commentary on our published study of ungulate behavioral responses to roads and traffic in South Africa (Mulero-Pazmany, D’Amico & Gonzalez-Suarez, 2016). We welcome the opportunity to engage in discussion regarding road ecology in Africa and how to best assess impacts and interpret findings. We all agree that understanding anthropogenic impacts, including those of roads and traffic, on wildlife and protected areas is important and that speculative inferences should be avoided. However, we find Jackson et al.’s criticism largely unsubstantiated and affected by statistical misconceptions and errors. We comment on the key points made by Jackson et al. (2017) below.

Statistical analysis of vertical accelerations of planing craft: Common pitfalls and how to avoid them

A careful review of the literature on the analysis of vertical accelerations of high-speed watercraft reveals that a number of statistical misconceptions are pervasive. These include: (1) ignoring vertical thresholding in analysis, (2) misunderstanding the relationships between the Exponential and other distributions, (3) failing to report uncertainty, and (4) failing to discuss alternative methods for estimation. This article brings these issues to the fore and shows how popular methods, when applied to simulated data, can give inaccurate results. We set forth proper methods for analysis and illustrate their use on a data set from a full-scale planing craft.

Response to Comment on "Estimating the reproducibility of psychological science".

Gilbert et al. conclude that evidence from the Open Science Collaboration's Reproducibility Project: Psychology indicates high reproducibility, given the study methodology. Their very optimistic assessment is limited by statistical misconceptions and by causal inferences from selectively interpreted, correlational data. Using the Reproducibility Project: Psychology data, both optimistic and pessimistic conclusions about reproducibility are possible, and neither are yet warranted.

Rejoinder to Letter to the Editor

We are certainly in agreement with the discussant that there are statistical misconceptions regarding the lottery that could have greater repercussions than the one regarding odds versus probabilities. Statistical terms which are misinterpreted according to use and custom (e.g., randomness, independence, causality, experiment, normal) are found in everyday use. It is in the sense of the generality of their use and the media throughout which these misconceptions “spread” that this situation could be classified as a “pandemic.”

Data interpretation in breath biomarker research: pitfalls and directions**This work was presented at the Breath Analysis Summit 2011 (11–14 September 2011, Parma, Italy).

Most—if not all—potential diagnostic applications in breath research involve different marker concentrations rather than unique breath markers which only occur in the diseased state. Hence, data interpretation is a crucial step in breath analysis. To avoid artificial significance in breath testing every effort should be made to implement method validation, data cross-testing and statistical validation along this process. The most common data analysis related problems can be classified into three groups: confounding variables (CVs), which have a real correlation with both the diseased state and a breath marker but lead to the erroneous conclusion that disease and breath are in a causal relationship; voodoo correlations (VCs), which can be understood as statistically true correlations that arise coincidentally in the vast number of measured variables; and statistical misconceptions in the study design (SMSD).CV: Typical confounding variables are environmental and medical history, host factors such as gender, age, weight, etc and parameters that could affect the quality of breath data such as subject breathing mode, effects of breath sampling and effects of the analytical technique itself.VC: The number of measured variables quickly overwhelms the number of samples that can feasibly be taken. As a consequence, the chances of finding coincidental ‘voodoo’ correlations grow proportionally. VCs can typically be expected in the following scenarios: insufficient number of patients, (too) many measurement variables, the use of advanced statistical data mining methods, and non-independent data for validation.SMSD: Non-prospective, non-blinded and non-randomized trials, a priori biased study populations or group selection with unrealistically high disease prevalence typically represent misconception of study design.In this paper important data interpretation issues are discussed, common pitfalls are addressed and directions for sound data processing and interpretation are proposed.

Key statistical and analytical issues for evaluating treatment effects in periodontal research

Statistics is an indispensible tool for evaluating treatment effects in clinical research. Due to the complexities of periodontal disease progression and data collection, statistical analyses for periodontal research have been a great challenge for both clinicians and statisticians. The aim of this article is to provide an overview of several basic, but important, statistical issues related to the evaluation of treatment effects and to clarify some common statistical misconceptions. Some of these issues are general, concerning many disciplines, and some are unique to periodontal research. We first discuss several statistical concepts that have sometimes been overlooked or misunderstood by periodontal researchers. For instance, decisions about whether to use the t-test or analysis of covariance, or whether to use parametric tests such as the t-test or its non-parametric counterpart, the Mann-Whitney U-test, have perplexed many periodontal researchers. We also describe more advanced methodological issues that have sometimes been overlooked by researchers. For instance, the phenomenon of regression to the mean is a fundamental issue to be considered when evaluating treatment effects, and collinearity amongst covariates is a conundrum that must be resolved when explaining and predicting treatment effects. Quick and easy solutions to these methodological and analytical issues are not always available in the literature, and careful statistical thinking is paramount when conducting useful and meaningful research.

Misconceptions in Descriptive Statistics Among Postgraduates in Social Sciences

Abstract Many efforts on identifying misconceptions among schools and undergraduate students have been done, but for postgraduates, in particular serviced teachers, are yet to be understood. Researches, thesis writing and publications require substantial level of statistical reasoning for postgraduates. The purpose of this study is to identify and analyze postgraduates’ misconceptions in descriptive statistics (central tendency and variability). The instrument was a set of questions design to test statistical reasoning in descriptive statistics. The test was administered to 122 postgraduates who are enrolling in a course on application of statistics in educational research which is a requirement for their master program in education. The misconceptions were analyzed based on the work of various researchers (Lee & Meletiou, 2003; Magina & Cazorla, 2008). This paper describes some similar misconceptions that were found by other researchers; however, some misconceptions on the concept of average were not seen among postgraduates. The overall results show that some postgraduates have poor statistical reasoning skills. This study suggests that statistical reasoning should be taught strategically so as to minimize misconceptions among postgraduate students as well as to empower their statistical reasoning ability.