Use Of Effect Sizes Research Articles

Multiple baseline and multiple probe design studies targeting academic skills: Trends over time in effect sizes

AbstractSingle‐case design (SCD) is a quantitative experimental technique in which participants serve as their own control. The use of an effect size in SCD allows evaluation of outcomes as well as comparison of outcomes via meta‐analyses. Characteristics of SCD research make the selection of an appropriate effect size complicated. Additionally, there are a number of factors that complicate the use of SCDs as a means to improve academic skills. The purpose of this study was to examine patterns by which SCD effect sizes are used to quantify outcomes from academic interventions. To do so, a descriptive analysis of an extant database of SCD studies was conducted. The authors created frequency tables for each effect size identified in the database as well as graphs to show the extent to which the use of effect sizes changed over time. The authors also determined whether the most frequently used effect sizes were appropriate for summarizing changes in academic outcomes. Although many effect sizes have been developed, only a small number are routinely used in SCD research for academic skills. The most frequently used effect sizes were those that come from standardized statistics, compared with those that utilized principles of regression or Bayesian analysis.

Is bilingualism linked to well-being? Evidence from a big-data survey

Abstract In applied linguistics generally and bilingualism research in particular, psychological variables remain a much under-investigated sub-category of individual differences compared with cognitive ones. To better understand the under-researched psychological effects of bilingualism, this study investigated well-being, a psychological construct, based on a big-data survey. Drawing upon a national survey (N = 12,582), we examined the influence of bilingualism (operationalised as foreign language (FL) proficiency) and 13 sociobiographical variables (e.g., socio-economic status, SES) on well-being. Among these 14 initial independent variables, perceived social fairness, SES, and health emerged as important predictors for well-being, with FL proficiency and national language (NL) proficiency as potentially important predictors; crucially, FL proficiency was more important than NL proficiency. As the first systematic attempt to link bilingualism with well-being, our study advocates (1) a more holistic perspective towards language (including NL and FL(s)) in any bilingual context and (2) fuller use of effect sizes.

Comparison of relative change with effect size metrics in Alzheimer’s disease clinical trials

BackgroundPer cent slowing of decline is frequently used as a metric of outcome in Alzheimer’s disease (AD) clinical trials, but it may be misleading. Our objective was to determine whether...

National identity and multilingualism: a survey of Chinese Mongolian university students

ABSTRACT In the field of multilingualism, limited research has explored the link between language proficiency and national identity, although the key role of language in the national identity (NI) formation has been acknowledged in other fields (e.g. sociology) over the past five decades. Aiming to narrow this gap, the present study examined Chinese Mongolian students’ NI and the influence from multilingualism (operationalised as self-rated proficiency in respectively Putonghua and English, their L2 and L3) and other sociobiographical variables (e.g. gender). NI was measured with a questionnaire item designed to gauge this construct that has been used in the latest waves of the Chinese Social Survey (CSS), a longitudinal nation-wide survey in China. The participants (N = 586) achieved a mean score of 4.26 (out of five) with the bootstrapped 95% confidence interval [4.20–4.32], indicating a high level of NI. Regression analyses showed that the two measures of multilingualism, attitudes respectively toward Putonghua, English, and trilingualism, and other sociobiographical variables influenced NI to different degrees. Calls for more comprehensive data concerning NI in multilingualism research and for more robust statistics (e.g. fuller use of effect size) were made. Policy implications vis-à-vis cultivating multilingualism among ethnic minorities were also discussed.

Remember to Leverage Effect Size Estimates

In 2023, the Methodology Corner column has opted to promote the efficacious use of effect sizes among our community of researchers. In this first entry of the year, we call attention to broad improvement in effect size reporting germane to our community, including more consistent reporting of effect sizes and their interpretations. [J Nurs Educ. 2023;62(2):116.].

Penggunaan Effect Size Sebagai Mediasi dalam Koreksi Efek Suatu Penelitian

This study aims to examine studies on the use of effect size in estimating the effect size of an experiment in various fields of science. The method used is Literature Review by searching for relevant articles in the databases of Google Scholar, National Center or Biotechnology Information (NCBI), Mendeley and ScienceDirect. The results of the study indicate that overall the articles reviewed show the use of effect size as a statistical test to determine the effect size of treatment in research studies. However, in its use, the effect size needs to be included with other statistical tests to obtain a small refractive index so that the data is more accurate. In conclusion, the effect size is part of a statistical test that plays a role in helping users understand the magnitude of the differences found in a research experiment.
 
 Keywords: Accuracy, Effect Size, Experiment, Statistics

Institutional Profiling for Educational Development: Identifying Which Conditions for Student Success to Address in a Given Educational Setting—A Case Study

Thanks to a wealth of research on college student retention, we have today a good understanding of the factors that facilitate student success. However, actions taken to promote student success are far from always effective. The teaching–learning environments differ between institutions and provide, to a greater or lesser extent, the conditions that contribute to student success. The purpose of this case study is to test a method, based on a questionnaire and the use of effect sizes, to establish a “profile” of a particular university department to identify to what extent five conditions for student success (institutional commitment, expectations, support, feedback, and student involvement) contributed to student success in courses given at this department. The practical implications of the profiling method used in the study are discussed.

Bayesian connective field modeling using a Markov Chain Monte Carlo approach

The majority of neurons in the human brain process signals from neurons elsewhere in the brain.Connective Field (CF) modelling is a biologically-grounded method to describe this essential aspect of the brain's circuitry. It allows characterizing the response of a population of neurons in terms of the activity in another part of the brain. CF modelling translates the concept of the receptive field (RF) into the domain of connectivity by assessing, at the voxel level, the spatial dependency between signals in distinct cortical visual field areas. Thus, the approach enables to characterize the functional cortical circuitry of the human cortex. While already very useful, the present CF modelling approach has some intrinsic limitations due to the fact that it only estimates the model's explained variance and not the probability distribution associated with the estimated parameters. If we could resolve this, CF modelling would lend itself much better for statistical comparisons at the level of single voxels and individuals. This is important when trying to gain a detailed understanding of the neurobiology and pathophysiology of the visual cortex, notably in rare cases. To enable this, we present a Bayesian approach to CF modeling (bCF). Using a Markov Chain Monte Carlo (MCMC) procedure, it estimates the posterior probability distribution underlying the CF parameters. Based on this, bCF quantifies, at the voxel level, the uncertainty associated with each parameter estimate. This information can be used in various ways to increase confidence in the CF model predictions. We applied bCF to BOLD responses recorded in the early human visual cortex using 3T fMRI. We estimated both the CF parameters and their associated uncertainties and show they are only weakly correlated. Moreover, we show how bCF facilitates the use of effect size (beta) as a data-driven parameter that can be used to select the most reliable voxels for further analysis. Finally, to further illustrate the functionality gained by bCF, we apply it to perform a voxel-level comparison of a single, circular symmetric, Gaussian versus a Difference-of-Gaussian model. We conclude that our bCF framework provides a comprehensive tool to study human functional cortical circuitry in health and disease.

Environmental pollution impacts: Are p values over-valued?

An examination revealed the dominance of the published literature of environmental science by p values. Meanwhile, the use of effect size has been neglected in publications reporting primary data, yet the size of effect is often more informative than p values inference in assessing the effects of pollution on living organisms, comparing susceptibility/resistance among organisms, and ranking pollutants according to their potency, among others. Statistical significance does not necessarily mean biological, practical, or scientific significance, and its use based on (often misinterpreted) p values reflects the average response or effect at average conditions based on an assumed linear model fit to the entire sample. However, pollution impacts and organismal responses are rarely characterized by linear and symmetric features, and dichotomous ‘statistical significance’ based on p values is inadequate to fully describe data and findings. Considering ‘the fallacy of the average’, variance, and differential response of different population percentiles in new studies would provide otherwise wasted biologically, practically, or scientifically significant information. Since p values often inform as to whether some findings warrant further examination, journals should consider mandating the reporting of effect sizes and confidence intervals, together with p values (should they be used), to provide more integrated information regarding pollution impacts. Moreover, replacing ‘statistical significance’ with language of evidence, especially in key components of publications, such as abstracts and conclusions, could help preventing potential misleading of the public and decision and policy makers.

Use of Effect Size in medical research: A brief primer on its why and how

Use of Effect Size in medical research: A brief primer on its why and how

Current use of effect size or confidence interval analyses in clinical and biomedical research.

The isolated use of the statistical hypothesis testing for two group comparison has limitations, and its combination with effect size or confidence interval analysis as complementary statistical tests is recommended. In the present work, we estimate the use of these complementary statistical tests (i.e. effect size or confidence interval) in recently published in research articles in clinical and biomedical areas. Methods: The ProQuest database was used to search published studies in academic journals between 2019 and 2020. The analysis was carried out using terms that represent five areas of clinical and biomedical research: “brain”, “liver”, “heart”, “dental”, and “covid-19”. A total of 119,558 published articles were retrieved. Results: The relative use of complementary statistical tests in clinical and biomedical publications was low. The highest frequency usage of complementary statistical tests was among articles that also used statistical hypothesis testing for two-sample comparison. Publications with the term “covid-19” showed the lowest usage rate of complementary statistical tests when all article were analyzed but presented the highest rate among articles that used hypothesis testing. Conclusion: The low use of effect size or confidence interval in two-sample comparison suggests that coordinate measures should be taken in order to increase the use of this analysis in clinical and biomedical research. Their use should be emphasized in statistical disciplines for college and graduate students, become a routine procedure in research laboratories, and recommended by reviewers and editors of scientific journals.Supplementary InformationThe online version contains supplementary material available at 10.1007/s11192-021-04150-3.

A review of CLIL outcomes: Calling for fuller use of effect sizes and less reliance uponp

This article reviews empirical quantitative studies concerning the outcomes of Content and Language Integrated Learning (CLIL) from the perspective of methodological rigour. This perspective advocates for fuller use of effect sizes (ESs) and less reliance upon the statistical significance (i.e., p). Regarding making fuller use of ESs, it proposes 1) reporting ESs for both statistically significant and non-significant results, 2) specifying which ES benchmark system for interpretation, 3) utilizing a topic-specific ES benchmark system, 4) valuing ES over p, and 5) interpreting ESs adequately. Regarding relying less on p, this article underscores 1) reporting exact p values and 2) using ‘statistically’ to modify ‘(non-)significant(ly)’. This review contributes to a better understanding of the inconsistent findings concerning the outcomes of CLIL. It also provides some suggestions for EAP practitioners (e.g., emphasizing reporting and interpreting ESs in their EAP teaching).

The Relative Age Effect in Czech U14 Male and Female Tennis Players in 2007–2016

The issue of the Relative Age Effect (RAE) has been long researched, discussed and published both in the academic and coaching community and the number of studies on it in various sports has significantly grown in recent years. The aim of this study was to verify the existence of RAE among Czech male (n = 6552) and female (n = 4131) junior tennis players and to identify possible differences in birthdate effect between male and female players. The research was carried out in players registered in the years 2007–2016 in the U14 age category in Czech Tennis Association (CTA) database; the athletes were divided into three subgroups (‘Ranked’, ‘Top 100’, ‘Top 10’). Research data were analysed by the methods of descriptive and inferential statistics: relative and absolute frequency, chi-square goodness of fit test () and chi-square test of independence () with the use of effect size (ES index w). A declining tendency of frequencies from Q1 to Q4 between male and female junior players was proven in all three subgroups.In the whole period of 2007–2016, a significant and strong RAE was demonstrated only in the ‘Top 10’ male subgroup (RAE was significant and ES was small or trivial in the other two subgroups). Among the female players, RAE was significant in all three subgroups (ES was small or trivial). Gender differences in RAE in favour of male players were significant in ‘Top 100’ and ‘Ranked’ (ES was small or trivial in all three subgroups). In the short and long term, RAE can have significant implications for the sport development of athletes; both coaches and the professional public can therefore be recommended to pay attention to this issue. The impact of RAE in sport, i.e. the uneven distribution of athletes’ birthdates, is more pronounced especially among junior athletes and often significantly affects their sports development and career.

Knee Pads Do Not Affect Physical Performance in Young Female Volleyball Players.

Knee pads have become increasingly popular among volleyball players. Given the fact high-intensity activities that are crucial to successfully playing this sport lead to an increased risk of a knee injury, the primary use of knee pads is to prevent potential injury. However, no research has been carried out to explain the effects of knee pads on the most important physical abilities in volleyball players, thus directly affecting performance. This study was undertaken to determine the effects of knee pads on the explosive power of the lower extremities, linear speed, and agility in young female volleyball players. In two separated sessions, 84 female volleyball players (age: 14.83 ± 0.72 years; height: 163.19 ± 8.38 cm; body mass: 53.64 ± 10.42 kg; VE: 5.30 ± 3.39 years) completed squat jumps (SJ), countermovement jumps (CMJ) with and without arm swing, linear sprints at 5-m and 10-m, modified t-test, and 5-10-5 shuttle test. Data analyses included descriptive statistics, paired sample T-tests and use of effect size (ES). There was no statistical difference between the two conditions for SJ (p = 0.156; ES = 0.18), CMJ (p = 0.817; ES = 0.03), CMJ with arm swing (p = 0.194; ES = 0.14), linear sprint at 5 m (p = 0.789; ES = 0.03) and 10 m (p = 0.907; ES = −0.01), modified t-test (p = 0.284; ES = 0.13), and 5-10-5 shuttle test (p = 0.144; ES = 0.19). Wearing knee pads has neither an inhibitory nor positive effects on explosive power of the lower extremities, linear speed, and agility in young female volleyball players.

A Metastudy of Journal of Counseling & Development: An Analysis of Publication Characteristics From 2000 to 2019

This metastudy evaluates trends and patterns in the author and article characteristics found in the Journal of Counseling & Development from 2000 to 2019. Special attention was given to research articles to more accurately identify and evaluate the research advancing the counseling profession. Findings suggest significant shifts in authorship (e.g., majority women authors, more international lead authors, 3.18 authors per article), content type (e.g., more research and international counseling articles and fewer practice, theory, trends, and profiles articles; slightly more counseling process articles; slightly fewer multicultural issues and health/well‐being articles), research design (i.e., more intervention articles; more true/quasi‐experimental, single‐subject research, and meta‐analytic designs and fewer correlational designs); participants (i.e., slightly larger sample sizes); and statistics used in studies (i.e., more basic or intermediate and fewer advanced statistics). Although the use of effect size and sample reliability estimates increased, the use of sample validity estimates did not.

The use of effect size in veterinary medicine

Effect size is a statistical index that measures the magnitude of the effect generated by the variable of interest in a study, in a sense, reflecting the practical or clinical value of the study in addition to the statistical results. In recent years, it has become preferable to report the effect size expressing practical significance in addition to the statistical significance expressed by the p-value in hypothesis tests in scientific research, and even it has been required by some scientific journals. By reporting the effect size, it is possible to use it in statistical power analysis, to compare the results of the studies, and to determine the amount of the effect in the study. In this study, by mentioning the concept of effect size, some effect size indices calculated according to research designs are introduced with examples of the field of veterinary medicine.

Inclusion of Effect Size Measures and Clinical Relevance in Research Papers.

There are multiple issues that arise when researchers focus on and only report "statistical significance" of study findings. An important element that is often not included in reports is a discussion of clinical relevance. The authors address issues related to significance, the use of effect sizes, confidence or credible intervals, and the inclusion of clinical relevance in reports of research findings. Measures of magnitude, precision, and relevance such as effect sizes, confidence intervals (CIs), and clinically relevant effects are described in detail. In addition, recommendations for reporting and evaluating effect sizes and CIs are included. Example scenarios are presented to illustrate the interplay of statistical significance and clinical relevance. There are several issues that may arise when significance is the focus of clinical research reporting. One issue is the lack of attention to nonsignificant findings in published works although findings show clinical relevance. Another issue is that significance is interpreted as clinical relevance. As well, clinically relevant results from small-sample studies are often not considered for publication, and thus, findings might not be available for meta-analysis. Findings in research reports should address effect sizes and clinical relevance and significance. Failure to publish clinically relevant effects and CIs may preclude the inclusion of clinically relevant studies in systematic reviews and meta-analyses, thereby limiting the advancement of evidence-based practice. Several accessible resources for researchers to generate, report, and evaluate measures of magnitude, precision, and relevance are included in this article.

Wrangling with p-values versus effect sizes to improve medical decision-making: A tutorial.

The most pervasive and damaging myth in clinical research is that the smaller the p-value, the stronger the hypothesis. In reality, the p-value primarily reflects the quality of research design decisions. The most common proposal to avoid misleading conclusions from clinical research requires the appropriate use of effect sizes, but which effect size, used when and how, is an open question. A solution is proposed for perhaps the most common problem in clinical research, the comparison between two populations, for example, comparison of two treatments in a randomized clinical trial or comparison of high risk versus low risk individuals in an epidemiological study: the success rate difference or equivalently the number needed to treat/take (NNT).

P a d (Používání statistické a věcné významnosti v českých sociálních vědách)

The aim of the article is to quantify how often in leading Czech social-science journals (Ceskoslovenska psychologie / Czechoslovak Psychology, Pedagogika/Pedagogy, and Sociologický casopis / Czech Sociological Review) authors choose the wrong procedures to analyse quantitative data. In particular, attention is focused on the incorrect choice of statistical tests, their misinterpretation and mechanical application, and the use of effect sizes, that are so highly recommended nowadays. The basic research period was ten years, from 2005 to 2014, and for the Czech Sociological Review the period was extended back to 1995. The results of the content analysis of published articles (N=363) show that statistical tests are applied quite often to data that are not suitable for statistical tests: this is found in about one-fifth of cases in Czech Sociological Review, one-half in Pedagogy, and more than three-quarters in Czechoslovak Psychology. In addition, authors often make mechanical use of statistical methods or make incorrect interpretations (in over 40% of articles in the Czech Sociological Review over the last 10 years) and there are rarely any substantive interpretations of results (especially in Czechoslovak Psychology). Effect sizes are applied relatively often, but there are also gaps in their usage. It is clear from the results that changes are necessary both in the teaching of quantitative methodology and publishing practices in this subject area.

Acute caffeine supplementation promotes small to moderate improvements in performance tests indicative of in-game success in professional female basketball players.

The aim of this study was to determine the effect of acute caffeine supplementation on anaerobic performance in professional female basketball players. A double-blind, placebo-controlled, experimental design was used in a randomized counterbalanced manner. In separate sessions, 10 professional basketball players ingested caffeine (3 mg/kg body mass) or a placebo (dextrose: 3 mg/kg body mass) 60 min before completing countermovement jumps (CMJ) with and without arm swing, a squat jump (SJ), the Lane Agility Drill, 20-m sprints (with 5-m and 10-m split times recorded) with and without dribbling a ball, and a suicide run. Participants provided ratings of perceived exertion (RPE) and ratings of perceived performance 30 min following testing. Data analyses included the use of effect size (ES) and significance. Caffeine supplementation produced small nonsignificant (p > 0.05) increases in CMJ without arm swing (ES = 0.30), CMJ with arm swing (ES = 0.29), SJ (ES = 0.33), and the lane agility drill (ES = -0.27). Caffeine supplementation produced small to moderate significant improvements in 10-m (ES = -0.63; p = 0.05) and 20-m (ES = -0.41; p = 0.04) sprint times without dribbling. Caffeine supplementation promoted a moderate significant reduction in RPE during the test battery (ES = -1.18; p = 0.04) and a small nonsignificant improvement in perceived performance (ES = 0.23; p = 0.53). Acute caffeine supplementation may produce small to moderate improvements in key performance attributes required for basketball while reducing RPE.