Effect Size Statistics Research Articles

Daycare Attendance, Stress, and Mental Health

Daycare stress can be indexed by cortisol, and elevated levels of cortisol have been implicated in the onset and development of mental health disorders. Our objective was to quantify the associations between daycare and cortisol and to identify individual and environmental conditions under which daycare attendance is associated with cortisol concentrations. We used Cohen effect size statistics to quantify these associations and to compare them across 11 published studies that were identified with MEDLINE and PsycINFO. Cortisol levels increased during the daycare day, whereas they decreased when children stayed at home. The mean effect size was d = 0.72. The magnitude of the daycare-stress relation seemed to vary under 3 specific conditions. First, the effect size was larger for children in low-quality daycare (d = 1.15), whereas there was essentially little or no effect for children in high-quality daycare (d = 0.10). Second, the effect size was larger for preschoolers (aged 39 to 59 months) (d = 1.17) than for infants (aged 3 to 16 months) (d = 0.11) or school-aged children (aged 84 to 106 months) (d = 0.09). Third, children with difficult temperaments in daycare were more likely to exhibit a rising pattern of cortisol, compared with children who were not difficult. Our review suggests that daycare attendance in relatively low-quality daycare conditions and for children with difficult temperaments may result in atypical cortisol elevation. Although the link between atypical cortisol elevation and mental health requires further study, programs aimed at improving the quality of daycare services during the preschool years are expected to lead to better physiological adaptation to daycare and to reduce the risks of mental health problems.

Rating scales in depression: limitations and pitfalls

Since the introduction of antidepressants to psychopharmacology in the 1960s, the Hamilton Depression Rating Scale (HAM-D) has been the most frequently used rating scale for depression. When used as a scale for prediction of outcome with antidepressants, the HAM-D, by its total score, has obtained limited use analogous to the Diagnostic and Statistical Manual of Mental Disorders. 4th ed (DSM-IV) diagnosis of major depression. Most research has been devoted to the use of the HAM-D to discriminate between placebo and active drugs or to show dose-response relationship in patients with major depression. An improvement in the total HAM-D score during a drug trial does not, however, in itself qualify the drug as an antidepressant, because the total score is not a sufficient statistic. The problem of statistical versus clinical significance when analyzing placebo-controlled trials, including dose-response relationship, is outlined, with the recommendation to use effect size statistics.

Inadequate Statistical Power of Negative Clinical Trials in Urological Literature

Negative studies provide valuable information. However, conducting studies with inadequate power is unethical and an inefficient use of resources. The purpose of this study was to determine the prevalence of negative studies with inadequate power in urological literature. The Journal of Urology, Urology and BJU International (formerly British Journal of Urology) from 1982 to 2002 were searched using the Ovid MEDLINE database. All clinical trials that contained the phrase "no difference" were identified. Data necessary for power calculation were extracted from applicable studies. Of the 417 articles identified in the MEDLINE search, 127 were negative studies that contained enough information to be analyzed. There were 70 (55%) articles from The Journal of Urology, 35 (28%) from BJU International and 22 (17%) from Urology. Of the studies that used continuous variables 65% had adequate power (greater than 80%) to detect a 50% difference between groups and 32% had adequate power to detect a 25% difference. Of the studies that used dichotomous variables only 33% had adequate power to detect a 50% difference between groups and 23% had adequate power to detect a 25% difference. Levels of adequate power in negative studies did not improve over time (p = 0.258). Many negative studies in urological literature are inconclusive because they lack adequate power to detect even large differences between groups. Inadequately powered studies often result in false conclusions that alter clinical behavior and deter further research. Therefore, it is imperative to consider power when interpreting literature. When designing future investigations power calculations should be performed to ensure sufficient patient recruitment to attain clinically meaningful results.

The Letter Contrast Sensitivity Test: Clinical Evaluation of a New Design

To compare the reliability, validity, and responsiveness of the Mars Letter Contrast Sensitivity (CS) Test to the Pelli-Robson CS Chart. One eye of 47 normal control subjects, 27 patients with open-angle glaucoma, and 17 with age-related macular degeneration (AMD) was tested twice with the Mars test and twice with the Pelli-Robson test, in random order on separate days. In addition, 17 patients undergoing cataract surgery were tested, once before and once after surgery. The mean Mars CS was 1.62 log CS (0.06 SD) for normal subjects aged 22 to 77 years, with significantly lower values in patients with glaucoma or AMD (P<0.001). Mars test-retest 95% limits of agreement (LOA) were +/-0.13, +/-0.19, and +/-0.24 log CS for normal, glaucoma, and AMD, respectively. In comparison, Pelli-Robson test-retest 95% LOA were +/-0.18, +/-0.19, and +/-0.33 log CS. The Spearman correlation between the Mars and Pelli-Robson tests was 0.83 (P<0.001). However, systematic differences were observed, particularly at the upper-normal end of the range, where Mars CS was lower than Pelli-Robson CS. After cataract surgery, Mars and Pelli-Robson effect size statistics were 0.92 and 0.88, respectively. The results indicate the Mars test has test-retest reliability equal to or better than the Pelli-Robson test and comparable responsiveness. The strong correlation between the tests provides evidence the Mars test is valid. However, systematic differences indicate normative values are likely to be different for each test. The Mars Letter CS Test is a useful and practical alternative to the Pelli-Robson CS Chart.

Aggregating Student Achievement Trends Across States With Different Tests: Using Standardized Slopes as Effect Sizes

The study of federal education initiatives that takes place over multiple years in multiple settings often calls for aggregating and comparing data-in particular, student achievement data-across a broad set of schools, districts, and states. The need to track the trends over time is complicated by the fact that the data from the different schools, districts, and states also may have been based on different achievement tests. This article suggests one approach for defining a common metric: calculating the standardized slope of a time series of datapoints. The standardized slope serves as an effect size statistic. Meta-analytic techniques can be employed because the student achievement test results from different states can therefore be considered equivalent to the findings from different studies.

Efficacy and Adverse Effects of Atypical Antipsychotics for Dementia: Meta-analysis of Randomized, Placebo-Controlled Trials

Atypical antipsychotic medications are widely used to treat delusions, aggression, and agitation in people with Alzheimer disease (AD) and other dementia. Several clinical trials have not shown efficacy, and there have been concerns about adverse events. The objective of this study was to assess the evidence for efficacy and adverse events of atypicals for people with dementia. MEDLINE, the Cochrane Register of Controlled Trials, meetings, presentations, and information obtained from sponsors were used in this study. Published and unpublished randomized, placebo-controlled, double-blind, parallel-group trials in patients with AD or dementia of atypical antipsychotics marketed in the United States were studied. Clinical and trials characteristics, outcomes, and adverse events were extracted. Data were checked by a second reviewer. Fifteen trials including 16 contrasts of atypical antipsychotics with placebo met selection criteria: aripiprazole (k = 3), olanzapine (k = 5), quetiapine (k = 3), and risperidone (k = 5). A total of 3,353 patients were randomized to drug and 1,757 to placebo. Standard meta-analysis methods were used to summarize outcomes. Quality of the reporting of trials varied. Efficacy on rating scales was observed by meta-analysis for aripiprazole and risperidone, but not for olanzapine. Response rates were frequently not reported. There were smaller effects for less severe dementia, outpatients, and patients selected for psychosis. Approximately one-third dropped out without overall differences between drug and placebo. Adverse events were mainly somnolence and urinary tract infection or incontinence across drugs, and extrapyramidal symptoms or abnormal gait with risperidone or olanzapine. Cognitive test scores worsened with drugs. There was no evidence for increased injury, falls, or syncope. There was a significant risk for cerebrovascular events, especially with risperidone; increased risk for death overall was reported elsewhere. Small statistical effect sizes on symptom rating scales support the evidence for the efficacy of aripiprazole and risperidone. Incomplete reporting restricts estimates of response rates and clinical significance. Dropouts and adverse events further limit effectiveness. Atypicals should be considered within the context of medical need and the efficacy and safety of alternatives. Individual patient meta-analyses are needed to better assess clinical significance and effectiveness.

A construct-driven investigation of gender differences in a leadership-role assessment center.

This study examined gender differences in a large-scale assessment center for officer entry in the British Army. Subgroup differences were investigated for a sample of 1,857 candidates: 1,594 men and 263 women. A construct-driven approach was chosen (a) by examining gender differences at the construct level, (b) by formulating a priori hypotheses about which constructs would be susceptible to gender effects, and (c) by using both effect size statistics and latent mean analyses to investigate gender differences in assessment center ratings. Results showed that female candidates were rated notably higher on constructs reflecting an interpersonally oriented leadership style (i.e., oral communication and interaction) and on drive and determination. These results are discussed in light of role congruity theory and of the advantages of using latent mean analyses.

Meta-analytic review of the effects of enrichment on stereotypic behavior in zoo mammals

This meta-analysis reports the effect enrichment has on the occurrence of stereotypic behavior exhibited by captive zoo mammals. The analysis also identifies which types of enrichment are most effective, which groups of animals benefit the most, and which types of stereotypes are most affected by environmental enrichment. The analysis included 54 studies that yielded 63 effect size statistics. Fifty-seven of sixty-three effect sizes went in the predicted direction (90%), with the animals participating in less stereotypic behavior during the enrichment condition than in the baseline condition. The mean effect size (correlation coefficient r) was 0.46. The combined P-value using both fixed and random effects methods was reveled to be <0.0000001. A file drawer N-value was calculated to identify the number of unretrieved studies (with a combined effect size of zero) that would be needed to nullify the results of this analysis. The file drawer N-value was 1,726, suggesting that it is highly unlikely that the significant results reported in this analysis are nullified by studies that remain in file drawers. Based on these results it was concluded that enrichment substantially reduces the frequency of stereotypic behavior exhibited by mammals living in zoo environments. Zoo Biol 0:1–21, 2006. © 2006 Wiley-Liss, Inc.

The Effectiveness of Volunteer Tutoring Programs: A Systematic Review

Synopsis Based on increasing demands for accountability in schools, school administrators are looking for ways to increase the academic performance of students including tutoring. Currently a variety of tutoring approaches are used including volunteer tutoring, peer tutoring, cross-age tutoring, and one-on-one tutoring. The evidence base related to volunteer tutoring has grown in recent years; consequently, this review is based upon only those evaluations of volunteer programs in which tutees were randomly assigned to a treatment or control group. We found 21 studies (with 27 different study cohorts in those studies because several studies provided separate reporting on multiple cohorts) reporting on randomized field trials to guide us in assessing the effectiveness of volunteer tutoring programs. Abstract Background Volunteer tutoring programs are intended to improve student performance, provide mentorship, and improve student self-esteem, as well as behavior. Despite the best of intention and effort, schools are not certain which volunteer tutoring programs are most effective. Therefore, we contend that a rigorous analysis of the extant literature regarding volunteer tutoring programs can provide schools with information about the most effective types of tutoring programs. Objectives To assess the effectiveness of volunteer tutoring programs for improving the academic skills of student enrolled in grades K-8 in the USA, and to further investigate for whom and under what conditions tutoring can be effective.

Development of a Portable Psychometric Testing Device for Use in the Field: An Alcohol Investigation

Cognitive and psychomotor performance have traditionally been assessed in the laboratory. There is a need for an objective portable assessment tool to assess cognitive and psychomotor performance. This study investigated the viability of a portable psychometric test battery, in a controlled laboratory environment, possibly leading to use in the field. A randomised, double-blind placebo controlled, three-way crossover design was employed. 16 subjects received 50 mg/100 ml and 80 mg/100 ml of alcohol and alcohol placebo. Performance was assessed with a tracking task, and an attention task presented on a small ruggedised handheld computer. The attention task showed no significant training effects; however, an element of the tracking task did. Statistical significance, effect size, and test-retest reliability analyses are presented indicating sensitivity of the portable psychometric test battery to the impairing effects of two separate doses of alcohol. Ability to undertake wide-scale impairment testing in the field with meaningful results in the absence of baseline data collection may have wide reaching implications, particularly in relation to the assessment of drivers impaired by drug use.

Comparative Validity Analysis of Multiple Measures of Violence Risk in a Sample of Criminal Offenders

This study compared the predictive validity of multiple indices of violence risk among 188 general population criminal offenders: Historical-Clinical-Risk Management-20 (HCR-20) Violence Risk Assessment Scheme, Violence Risk Appraisal Guide (VRAG), Violent Offender Risk Assessment Scale (VORAS), Hare Psychopathy Checklist-Revised (PCL-R), and Screening Version (PCL:SV). Several indices were related to violent recidivism with large statistical effect sizes: HCR-20 (Total, Clinical and Risk Management scales, structured risk judgments), VRAG, and behavioral scales of psychopathy measures. Multivariate analyses showed that HCR-20 indices were consistently related to violence and that the VRAG entered some analyses. Findings are inconsistent with a position of strict actuarial superiority, as HCR-20 structured risk judgments—an index of structured professional or clinical judgment—were as strongly related to violence.

Recommended effect size statistics for repeated measures designs

Investigators, who are increasingly implored to present and discuss effect size statistics, might comply more often if they understood more clearly what is required. When investigators wish to report effect sizes derived from analyses of variance that include repeated measures, past advice has been problematic. Only recently has a generally useful effect size statistic been proposed for such designs: generalized eta squared (eta2G; Olejnik & Algina, 2003). Here, we present this method, explain that eta2G preferred to eta squared and partial eta squared because it provides comparability across between-subjects and within-subjects designs, show that it can easily be computed from information provided by standard statistical packages, and recommend that investigators provide it routinely in their research reports when appropriate.

Effective Measurement of the Functional Progress of Stroke Clients

There is a need for occupational therapists to establish a range of outcome measures that can measure change effectively. The aim of this study was to identify effective methods of measuring the functional performance of individuals who had experienced a stroke and had been discharged from a rehabilitation ward. Measures with recorded validity and reliability and in clinical use were selected. The measures were the Barthel Index (Shah version), Nottingham Extended Activities of Daily Living scale, Motricity Index (Upper and Lower Limb), Rivermead Mobility Index and Nine-Hole Peg Test. Assessments were carried out at the point of discharge from hospital and 6 months later. Responsiveness was analysed using effect size and relative efficiency statistics. The practicality (feasibility) was established by comparing completion rates. The Nottingham Extended Activities of Daily Living scale and the Rivermead Mobility Index were found to be more responsive than the Barthel Index and Lower Limb Motricity Index respectively and to be practical after discharge from hospital. The Nine-Hole Peg Test was found to be more responsive than the Upper Limb Motricity Index but less practical. The analysis suggested that some measures are more suitable than others for tracking change in function after discharge from hospital for a wide case mix of people who have experienced a stroke.

Physical Activity And Psychological Health In Overweight And Obese Adults

PURPOSE To examine psychological health characteristics of active and inactive overweight and obese adults. METHODS The Profile of Mood States (POMS), Rosenberg self-esteem scale (RSE) and a Satisfaction with Life scale (SWL) were administered to 13 overweight or obese adults (age=41.3 ± 6.3y, BMI = 34.6 ± 5.8 kg/m2). Usual daily ambulatory activity was assessed with a pedometer (PED, YAMAX SW-200) and a pedometer log for 7-days. Average steps/day were calculated. Participants were categorized into high active (HA) or low active (LA) groups based on their pedometer scores. Independent samples t-test and effect size (Cohen's d) statistics were calculated. RESULTS The HA group accumulated significantly more steps (13,084 ± 1287 steps/day) than the LA group (4028 ± 1088 steps/day) (t(1 1)=−13.64, p <.001). HA participants had significantly higher scores on RSE (t(11)=−2.83, p <.05) and SWL (t(11)=−2.85, p <.05) scales than LA participants. Although no significant differences were found, examination of mean scores across all POMS subscales indicate a trend toward lower depression, tension, fatigue, and confusion as well as higher vigor for HA vs. LA participants. Effect size statistics (Cohen's d) indicate a large effect of classification (HA vs. LA) on differences in depression (.65), RSE (1.32) and SWL (1.72) scores. Moderate effect size statistics were observed for activity classification (HA vs LA) on differences observed in vigor (.42) and confusion (.58) ratings between groups. CONCLUSIONS These preliminary results indicate that the patterns of psychological benefits associated with higher PA levels among obese/overweight adults are similar to those found in research with a general population. Research has indicated a high prevalence of depression, low-self esteem, and otherwise poor psychological health factors among obese individuals. As such, one implication of these findings is that increasing accumulated ambulatory activity may contribute to improvements in psychological health, which in turn could reduce mood and self-esteem related barriers to beginning a more intense exercise or weight loss program. Supported by the UWM CHS SEED Grant Program (Swartz).

Physical Activity And Psychological Health In Overweight And Obese Adults

PURPOSE To examine psychological health characteristics of active and inactive overweight and obese adults. METHODS The Profile of Mood States (POMS), Rosenberg self-esteem scale (RSE) and a Satisfaction with Life scale (SWL) were administered to 13 overweight or obese adults (age=41.3 ± 6.3y, BMI = 34.6 ± 5.8 kg/m2). Usual daily ambulatory activity was assessed with a pedometer (PED, YAMAX SW-200) and a pedometer log for 7-days. Average steps/day were calculated. Participants were categorized into high active (HA) or low active (LA) groups based on their pedometer scores. Independent samples t-test and effect size (Cohen's d) statistics were calculated. RESULTS The HA group accumulated significantly more steps (13,084 ± 1287 steps/day) than the LA group (4028 ± 1088 steps/day) (t(1 1)=−13.64, p <.001). HA participants had significantly higher scores on RSE (t(11)=−2.83, p <.05) and SWL (t(11)=−2.85, p <.05) scales than LA participants. Although no significant differences were found, examination of mean scores across all POMS subscales indicate a trend toward lower depression, tension, fatigue, and confusion as well as higher vigor for HA vs. LA participants. Effect size statistics (Cohen's d) indicate a large effect of classification (HA vs. LA) on differences in depression (.65), RSE (1.32) and SWL (1.72) scores. Moderate effect size statistics were observed for activity classification (HA vs LA) on differences observed in vigor (.42) and confusion (.58) ratings between groups. CONCLUSIONS These preliminary results indicate that the patterns of psychological benefits associated with higher PA levels among obese/overweight adults are similar to those found in research with a general population. Research has indicated a high prevalence of depression, low-self esteem, and otherwise poor psychological health factors among obese individuals. As such, one implication of these findings is that increasing accumulated ambulatory activity may contribute to improvements in psychological health, which in turn could reduce mood and self-esteem related barriers to beginning a more intense exercise or weight loss program. Supported by the UWM CHS SEED Grant Program (Swartz).

Functionality or aesthetics?A pilot study of music therapy in the treatment of multiple sclerosis patients

Neuro-degenerative diseases are, and will remain, an enormous public health problem. Interventions that could delay disease onset even modestly will have a major public health impact. The aim of this study is to see which components of the illness are responsive to change when treated with music therapy in contrast to a group of patients receiving standard medical treatment alone. Twenty multiple sclerosis patients (14 female, 6 male) were involved in the study, their ages ranging from 29 to 47 years. Ten participants formed the therapy group, and 10 the matched control group matched by age, gender and the standard neurological classification scheme Expanded Disability Status Scale (EDSS). Exclusion criteria were pregnancy and mental disorders requiring medication. Patients in the therapy group received three blocks of music therapy in single sessions over the course of the one-year project (8-10 sessions, respectively). Measurements were taken before therapy began (U1), and subsequently every three months (U2-U4) and within a 6-month follow-up without music therapy (U5) after the last consultation. Test battery included indicators of clinical depression and anxiety (Beck Depression Inventory and Hospital Anxiety and Depression Scale), a self-acceptance scale (SESA) and a life quality assessment (Hamburg Quality of Life Questionnaire in Multiple Sclerosis). In addition, data were collected on cognitive (MSFC) and functional (EDSS) parameters. There was no significant difference between the music-therapy treatment group and the control group. However, the effect size statistics comparing both groups show a medium effect size on the scales measuring self-esteem (d, 0.5423), depression HAD-D (d, 0.63) and anxiety HAD-A (d, 0.63). Significant improvements were found for the therapy group over time (U1-U4) in the scale values of self-esteem, depression and anxiety. In the follow-up, scale values for fatigue, anxiety and self-esteem worsen within the group treated with music therapy. A therapeutic concept for multiple sclerosis, which includes music therapy, brings an improvement in mood, fatigue and self-acceptance. When music therapy is removed, then scale scores worsen and this appears to intimate that music therapy has an influence.

Determining sexual dimorphism in frog measurement data: integration of statistical significance, measurement error, effect size and biological significance

Several analytic techniques have been used to determine sexual dimorphism in vertebrate morphological measurement data with no emergent consensus on which technique is superior. A further confounding problem for frog data is the existence of considerable measurement error. To determine dimorphism, we examine a single hypothesis (Ho = equal means) for two groups (females and males). We demonstrate that frog measurement data meet assumptions for clearly defined statistical hypothesis testing with statistical linear models rather than those of exploratory multivariate techniques such as principal components, correlation or correspondence analysis. In order to distinguish biological from statistical significance of hypotheses, we propose a new protocol that incorporates measurement error and effect size. Measurement error is evaluated with a novel measurement error index. Effect size, widely used in the behavioral sciences and in meta-analysis studies in biology, proves to be the most useful single metric to evaluate whether statistically significant results are biologically meaningful. Definitions for a range of small, medium, and large effect sizes specifically for frog measurement data are provided. Examples with measurement data for species of the frog genus Leptodactylus are presented. The new protocol is recommended not only to evaluate sexual dimorphism for frog data but for any animal measurement data for which the measurement error index and observed or a priori effect sizes can be calculated.

Evaluating the Replicability of Sample Results: A Tutorial of Double Cross-Validation Methods

Although replication is important to the validity of a study and is endorsed by more and more scholars, few researchers in kinesiology attend to this issue. Some researchers may believe that statistical significance and effect size are the most important statistical issues in their research and thereby may have ignored the importance of result replication. Both significance testing and effect size, however, rarely inform researchers about the likelihood that results will be replicated in different samples drawn from the same population. The primary purpose of this tutorial is to provide readers with information regarding result replicability evaluation, focusing on double cross-validation approaches. This tutorial can also serve as a practical guide for conducting replicability analyses using double cross-validation methods.

Evaluating Stream Habitat Survey Data and Statistical Power Using an Example from Southeast Alaska

Stream habitat surveys and watershed assessments have been developed and used as monitoring tools for decades. Most rely on type I error as the primary criterion, with minor consideration of statistical power and effect size. We test for statistical differences in fish habitat condition between harvested and nonharvested watersheds from habitat survey data collected in southeast Alaska. We apply statistical power analysis to judge whether nonsignificant results can be interpreted with confidence. None of the fish habitat variables we examined were significant at α = 0.05; however, several P-values were less than 0.10 and consistent differences between harvested and nonharvested reaches were observed among channel types. Statistical power is low and the probability of not detecting differences is high when the effect size, scaled to the standard deviation of the measurement, is small to medium. For large effect sizes, the ability to detect differences was greater but did not exceed 85% for any measurement. Statistical power, effect size, and biological significance of the outcome are important considerations when the results are interpreted and can lend additional information to managers making decisions with data that are less than perfect.

Oral Health, Atherosclerosis, and Cardiovascular Disease

During the last two decades, there has been an increasing interest in the impact of oral health on atherosclerosis and subsequent cardiovascular disease (CVD). The advent of the inflammation paradigm in coronary pathogenesis stimulated research in chronic infections caused by a variety of micro-organisms-such as Chlamydia pneumoniae, Helicobacter pylori, and cytomegalovirus-as well as dental pathogens, since these chronic infections are thought to be involved in the etiopathogenesis of CVD by releasing cytokines and other pro-inflammatory mediators (e.g., C-reactive protein [CRP], tumor necrosis factor [TNF-alpha]) that may initiate a cascade of biochemical reactions and cause endothelial damage and facilitate cholesterol plaque attachment. Yet, due to the multi-factorial nature of dental infection and CVD, confirming a causal association is difficult, and the published results are conflicting. The main deficit in the majority of these studies has been the inadequate control of numerous confounding factors, leading to an overestimation and the imprecise measurement of the predictor or overadjustment of the confounding variables, resulting in underestimation of the risks. A meta-analysis of prospective and retrospective follow-up studies has shown that periodontal disease may increase the risk of CVD by approximately 20% (95% confidence interval [CI], 1.08-1.32). Similarly, the reported risk ratio between periodontal disease and stroke is even stronger, varying from 2.85 (CI 1.78-4.56) to 1.74 (CI 1.08-2.81). The association between peripheral vascular disease and oral health parameters has been explored in only two studies, and the resultant relative risks among individuals with periodontitis were 1.41 (CI 1.12-1.77) and 2.27 (CI 1.32-3.90), respectively. Overall, it appears that periodontal disease may indeed contribute to the pathogenesis of cardiovascular disease, although the statistical effect size is small.