Flaws In Study Design Research Articles

Retracting Inconclusive Research: Lessons from the Séralini GM Maize Feeding Study.

In September 2012, Gilles-Eric Séralini and seven coauthors published an article in Food and Chemical Toxicology claiming that rats fed Roundup©-resistant genetically modified maize alone, genetically modified maize with Roundup©, or Roundup© for 2 years had a higher percentage of tumors and kidney and liver damage than normal controls. Shortly after this study was published, numerous scientists and several scientific organizations criticized the research as methodologically and ethically flawed. In January 2014, the journal retracted the article without the authors' consent on the grounds that the research was inconclusive. In June 2014, Environmental Sciences Europe published a slightly modified version of the retracted paper. The publication, retraction and subsequent republication of the Séralini study raise important scientific and ethical issues for journal editors. Decisions to retract an article should be made on the basis of well-established policies. Articles should be retracted only for serious errors that undermine the reliability of the data or results, or for serious ethical lapses, such as research misconduct or mistreatment of animal or human subjects. Inconclusiveness, by itself, is not a sufficient reason for retracting an article, though a flawed study design might be. Retracted articles that are submitted for republication should undergo scientific review to ensure that they meet appropriate standards. Republished articles should be linked to the original, retracted publication. Journals that are reviewing studies with significant scientific and social implications should take special care to ensure that peer review is rigorous and fair.

Chinese patent medicines for the treatment of the common cold: a systematic review of randomized clinical trials.

BackgroundMany Chinese patent medicines (CPMs) have been authorized by the Chinese State of Food and Drug Administration for the treatment of the common cold. A number of clinical trials have been conducted and published. However, there is no systematic review or meta-analysis on their efficacy and safety for the common cold to justify their clinical use.MethodsWe searched CENTRAL, MEDLINE, EMBASE, SinoMed, CNKI, VIP, China Important Conference Papers Database, China Dissertation Database, and online clinical trial registry websites for published and unpublished randomized clinical trials (RCTs) of CPMs for the common cold till 31 March 2013. Revman 5.2 software was used for data analysis with effect estimate presented as relative risk (RR) and mean difference (MD) with a 95% confidence interval (CI).ResultsA total of five RCTs were identified. All of the RCTs were of high risk of bias with flawed study design and poor methodological quality. All RCTs included children aged between 6 months to 14 years. Results of individual trials showed that Shuanghuanglian oral liquid (RR 4.00; 95% CI: 2.26 to 7.08), and Xiaoer Resuqing oral liquid (RR 1.43; 95% CI: 1.15 to 1.77) had higher cure rates compared with antivirus drugs. Most of the trials did not report adverse events, and the safety of CPMs was still uncertain.ConclusionsSome CPMs showed a potential positive effect for the common cold on cure rate. However, due to the poor methodology quality and the defects in the clinical design of the included RCTs, such as the lack of placebo controlled trials, the inappropriate comparison intervention and outcome measurement, the confirmative conclusions on the beneficial effect of CPMs for the common cold could not be drawn.

Lessons from the infuse trials: do we need a classification of bias in scientific publications and editorials?

The original 13 Food and Drug Administration industry-sponsored recombinant human bone morphogenetic protein-2 (rhBMP-2) trials investigating its use in spinal fusion all reported no associated adverse events. However, subsequent series of studies began reporting complication rates that were much higher than those that were initially published. Critical analysis of the original rhBMP-2 industry-associated data found systematic alignment favoring positive outcomes with no proven clinical advantage over bone graft. The sources of potential bias leading to inaccurate reporting of original rhBMP-2 efficacy and safety profile include flawed study design, methodological technique, data reporting and analysis, and significant financial conflict of interest. As such, to ensure the integrity of the scientific literature, further measures should be taken by researchers, surgeons, authors, journal editors and reviewers to assess for potential sources of bias.

Renal artery stenosis: epidemiology and treatment.

Renal artery stenosis (RAS) is a frequently encountered problem in clinical practice. The disease encompasses a broad spectrum of pathophysiologies and is associated with three major clinical syndromes: ischemic nephropathy, hypertension, and destabilizing cardiac syndromes. The two most common etiologies are fibromuscular dysplasia and atherosclerotic renal artery disease with atherosclerotic disease accounting for the vast majority of cases. Atherosclerotic renovascular disease has considerable overlap with atherosclerotic disease elsewhere and is associated with a poor prognosis. A wide range of diagnostic modalities and treatment approaches for RAS are available to clinicians, and with the advent of endovascular interventions, selecting the best course for a given patient has only grown more challenging. Several clinical trials have demonstrated some benefit with revascularization but not to the extent that many had hoped for or expected. Furthermore, much of the existing data is only marginally useful given significant flaws in study design and inherent bias. There remains a need for further identification of subgroups and appropriate indications in hopes of maximizing outcomes and avoiding unnecessary procedures in patients who would not benefit from treatment. In recent decades, the study of RAS has expanded and evolved rapidly. In this review, we will attempt to summarize the amassed body of literature with a focus on the epidemiology of RAS including prevalence, overlap with other atherosclerotic disease, and prognosis. We will also outline existing diagnostic and treatment approaches available to clinicians as well as summarize the findings of several major clinical trials. Finally, we will offer our perspective on future directions in the field.

Calcium Supplementation: Is Protecting Against Osteoporosis Counter to Protecting against Cardiovascular Disease?

Calcium is the dominant mineral in bone and is a shortfall nutrient in the diet. For those consuming inadequate dietary calcium, calcium supplements have been a standard strategy for prevention of osteoporosis. Recently, calcium supplementation has been linked to both increased and decreased cardiovascular disease risk creating considerable uncertainty. Moreover, recent reports have shed uncertainty over the effectiveness of calcium supplements to reduce risk of fracture. The evidence for calcium supplementation effects to both reduce risk of fracture and increase coronary heart disease and mortality are reviewed. Although the importance of good calcium nutrition is well known, determining the advantage of calcium supplementation to either bone or heart health has been hampered by poor subject compliance and study design flaws. At present, the current Recommended Dietary Allowances for calcium still appear to be a good target with potential risks for chronic disease if intakes fall too short or greatly exceed these recommendations.

Commentary on Mactieret al. (2014): Methadone-assisted treatment and the complexity of influences on fetal development

Commentary on Mactier<i>et al</i>. (2014): Methadone-assisted treatment and the complexity of influences on fetal development

Does Male Circumcision Affect Sexual Function, Sensitivity, or Satisfaction?—A Systematic Review

Circumcision of males is commonly carried out worldwide for reasons of health, medical need, esthetics, tradition, or religion. Whether circumcision impairs or improves male sexual function or pleasure is controversial. The study aims to conduct a systematic review of the scientific literature. A systematic review of published articles retrieved using keyword searches of the PubMed, EMBASE, and Cochrane databases was performed. The main outcome measure is the assessment of findings in publications reporting original data relevant to the search terms and rating of quality of each study based on established criteria. Searches identified 2,675 publications describing the effects of male circumcision on aspects of male sexual function, sensitivity, sensation, or satisfaction. Of these, 36 met our inclusion criteria of containing original data. Those studies reported a total of 40,473 men, including 19,542 uncircumcised and 20,931 circumcised. Rated by the Scottish Intercollegiate Guidelines Network grading system, 2 were 1++ (high quality randomized controlled trials) and 34 were case-control or cohort studies (11 high quality: 2++; 10 well-conducted: 2+; 13 low quality: 2-). The 1++, 2++, and 2+ studies uniformly found that circumcision had no overall adverse effect on penile sensitivity, sexual arousal, sexual sensation, erectile function, premature ejaculation, ejaculatory latency, orgasm difficulties, sexual satisfaction, pleasure, or pain during penetration. Support for these conclusions was provided by a meta-analysis. Impairment in one or more parameters was reported in 10 of the 13 studies rated as 2-. These lower-quality studies contained flaws in study design (11), selection of cases and/or controls (5), statistical analysis (4), and/or data interpretation (6); five had multiple problems. The highest-quality studies suggest that medical male circumcision has no adverse effect on sexual function, sensitivity, sexual sensation, or satisfaction.

Public Health or Private Profit?

We think the article by Sarrel et al.1 falls far short of the Journal's standards. While the post hoc analysis of the Women’s Health Initiative (WHI) results that Sarrel et al. employ is a legitimate tool for hypothesis generation, the authors present their conclusions as far more certain than that and even go so far as to recommend that their findings should inform clinical counseling. Given this, we were not surprised to read in the Acknowledgments that the lead author had been a paid consultant for a company that makes an estrogen therapy patch. We were, however, shocked and disappointed to see the nation’s leading public health journal provide a platform for such a misleading presentation of the current state of knowledge about use of estrogen therapy to prevent disease. This article purports to estimate the number of excess deaths caused by avoidance of estrogen, following the release of the WHI results. The authors ignored, however, the inconvenient fact that the WHI found no protective cardiovascular benefit for estrogen-alone compared with placebo.2 Instead, their analysis relies on an estimate derived from a post hoc subgroup analysis of the WHI, suggesting that estrogen may have a preventive health benefit for hysterectomized women aged 50 to 59 years. They then used this post hoc finding as the foundation for a mathematical model estimating how many women died because they did not take estrogen. Furthermore, the authors make claims about the benefits of estrogen that are not supported by evidence, and in some cases not even supported by a citation. For example, they state that “transdermal estradiol has been reported to be more effective than are oral estrogens in preventing cardiovascular events.”1(p1587) In addition, they never mention the fact that the WHI estrogen-only trial was stopped early because it found an increased risk of stroke and blood clots. Consequently, the authors’ conclusion that “decline in ET use since 2002 has resulted in a significant increase in mortality for hysterectomized women aged 50 to 59 years.”1(p1587) is based on a flawed study design and significantly misrepresents the state of the evidence on the health effects of taking estrogen. And because reporters rely on the Journal as a reliable source of health information, this misleading estimate was reported as a proven fact.3 We have come to expect drug company sponsored researchers to put forward sensationalistic and misleading claims about the benefits of estrogen, but we expect and deserve better from the Journal.

A Critical Evaluation of Migraine Trigger Site Deactivation Surgery

Migraine headache trigger site deactivation surgery is a term that encompasses 4 different surgical procedures that are performed based on headache onset location for the preventative treatment of migraine headaches. Multiple studies have demonstrated some efficacy of these procedures, but closer evaluation of the methodology of these studies reveals major flaws in study design. In this article, the author provides an overview of the procedures and presurgical screening tools, as well as a critical evaluation of 2 of the major studies that have been published. In addition, the author provides his opinion on future study designs that may help to better determine the potential efficacy of these experimental procedures and potential headache subtypes (contact point headache, supraorbital neuralgia, and occipital neuralgia) that may respond to peripheral decompression surgery.

Unsubstantiated concerns over the safety of use of sulphonylureas and insulin for increased risk of diabetes complications (使用磺脲类药物和胰岛素增加糖尿病并发症的担心并无事实根据)

Unsubstantiated concerns over the safety of use of sulphonylureas and insulin for increased risk of diabetes complications (使用磺脲类药物和胰岛素增加糖尿病并发症的担心并无事实根据)

Guidelines for the Management of Acute Cervical Spine and Spinal Cord Injuries

Guidelines for the Management of Acute Cervical Spine and Spinal Cord Injuries

Vacuum Assist: Angel or Demon CON

Vacuum-assisted venous drainage (VAVD) to enhance venous return during cardiopulmonary bypass (CPB) was described as early as 1958 but was not widely used until the late 1990s. VAVD was initially used to facilitate the use of smaller cannulas with ministernotomy but was increasingly used to allow reduction in CPB circuit size while maintaining CPB flow rates. This innovation was made without planned consideration to CPB circuit design, most critically that of the venous reservoir. Clinical reports of prime reduction facilitated by VAVD in both adult and pediatric CPB were associated with reduced nadir hematocrit and rates of transfusion that encouraged the proliferation of VAVD in CPB. Concomitantly, investigators have reported both in vitro and in vivo significantly increased arterial emboli associated with the use of VAVD, mostly related to accelerated venous air entrainment. In vitro studies continue to confirm this association and likely underestimate the resulting embolic load as a result of flawed study design. While the evidence for VAVD is equivocal, our understanding of the clinical impact of gaseous microemboli in CPB is similarly limited, most likely confined to blood–brain barrier disruption. It is only after two decades that CPB component design is receiving serious attention in terms of air handling. The ethics of innovation in the field of CPB warrant careful consideration. The application of VAVD is not without consequence.

Shoulder pain in primary care: diagnostic accuracy of clinical examination tests for non-traumatic acromioclavicular joint pain

BackgroundDespite numerous methodological flaws in previous study designs and the lack of validation in primary care populations, clinical tests for identifying acromioclavicular joint (ACJ) pain are widely utilised without concern for such issues. The aim of this study was to estimate the diagnostic accuracy of traditional ACJ tests and to compare their accuracy with other clinical examination features for identifying a predominant ACJ pain source in a primary care cohort.MethodsConsecutive patients with shoulder pain were recruited prospectively from primary health care clinics. Following a standardised clinical examination and diagnostic injection into the subacromial bursa, all participants received a fluoroscopically guided diagnostic block of 1% lidocaine hydrochloride (XylocaineTM) into the ACJ. Diagnostic accuracy statistics including sensitivity, specificity, predictive values, positive and negative likelihood ratios (LR+ and LR-) were calculated for traditional ACJ tests (Active Compression/O’Brien’s test, cross-body adduction, localised ACJ tenderness and Hawkins-Kennedy test), and for individual and combinations of clinical examination variables that were associated with a positive anaesthetic response (PAR) (P≤0.05) defined as 80% or more reduction in post-injection pain intensity during provocative clinical tests.ResultsTwenty two of 153 participants (14%) reported an 80% PAR. None of the traditional ACJ tests were associated with an 80% PAR (P<0.05) and combinations of traditional tests were not able to discriminate between a PAR and a negative anaesthetic response (AUC 0.507; 95% CI: 0.366, 0.647; P>0.05). Five clinical examination variables (repetitive mechanism of pain onset, no referred pain below the elbow, thickened or swollen ACJ, no symptom provocation during passive glenohumeral abduction and external rotation) were associated with an 80% PAR (P<0.05) and demonstrated an ability to accurately discriminate between an PAR and NAR (AUC 0.791; 95% CI 0.702, 0.880; P<0.001). Less than two positive clinical features resulted in 96% sensitivity (95% CI 0.78, 0.99) and a LR- 0.09 (95% CI 0.02, 0.41) and four positive clinical features resulted in 95% specificity (95% CI 0.90, 0.98) and a LR+ of 4.98 (95% CI 1.69, 13.84).ConclusionsIn this cohort of primary care patients with predominantly subacute or chronic ACJ pain of non-traumatic onset, traditional ACJ tests were of limited diagnostic value. Combinations of other history and physical examination findings were able to more accurately identify injection-confirmed ACJ pain in this cohort.

COUNTERPOINT PIECE: THE CASE FOR VARIETY IN CORRECTIVE FEEDBACK RESEARCH

Goo and Mackey (this issue) outline several apparent design flaws in studies that have compared the impact of different types of corrective feedback (CF). Furthermore, they argue that SLA researchers should stop comparing recasts to other types of CF because they are inherently different kinds of phenomena. Our response to their article addresses (a) the claim that the recast-learning relationship has been “settled,” (b) the misleading representation of our views on uptake, (c) the characterization of the CF comparison studies as being weak and invalid, and (d) Goo and Mackey’s recommendations concerning the most appropriate approach to investigating the effect of feedback on second language learning.

Complementary and Alternative Medicines Use during Pregnancy: A Systematic Review of Pregnant Women and Healthcare Professional Views and Experiences

Aims. To undertake a systematic review of the recent (2008–2013) primary literature, describing views and experiences of CAM use during pregnancy by women and healthcare professionals. Method. Medline, Cumulative Index to Nursing and Allied Health Literature, Cochrane Database of Systematic Review Library and Allied, and Complementary Medicine Database were searched. Studies reporting systemic CAM products (homeopathic preparations, herbal medicines, Vitamins and minerals, homeopathy, and special diets) alone or in combination with other nonsystemic CAM modalities (e.g., acupuncture) were included. Results. Database searches retrieved 2,549 citations. Removal of duplicates followed by review of titles and abstracts yielded 32 relevant studies. Twenty-two reported the perspectives of women and their CAM use during pregnancy, while 10 focused on healthcare professionals. The majority of studies had significant flaws in study design and reporting, including a lack of appropriate definitions of CAM and associated modalities, absence of detailed checklists provided to participants, the use of convenience sampling, and a general lack of scientific robustness in terms of data validity and reliability. Conclusion. To permit generalisability of study findings, there is an urgent need to expand the evidence base assessing CAMs use during pregnancy using appropriately designed studies.

The lessons of failure

The lessons of failure

Effectiveness of the bipolar sealer in reducing blood loss in total knee arthroplasty

Summary Total knee arthroplasty (TKA) is a fairly common procedure in the United States. Blood loss in the perioperative period is inevitable. Postoperative hematomas and hemarthrosis can develop when bleeding occurs into the knee joint postoperatively, thereby increasing pain and swelling and hindering recovery (Rosenberg, 2007). To minimize the loss of blood intraoperatively, conventional electrocautery is routinely used. However, an alternative approach in preventing blood loss intraoperatively involves the use of a bipolar sealer device designed to seal rather than cauterize blood vessels. Because the bipolar sealer operates at a cooler temperature than the traditional electrocautery method, researchers propose that the bipolar sealer is less traumatizing to tissue and minimizes blood loss in the perioperative period. Therefore, the objective of this study is to assess the effectiveness of the bipolar sealer when compared to the standard electrocautery device in terms of minimizing blood loss and preserving the patient's hemoglobin. A reduced need to transfuse blood could minimize patient risk related to blood transfusion and ultimately reduce hospital length of stay. The research study offers a retrospective collection of demographic, clinical, and outcome data from patients who underwent TKA either with standard electrocautery or bipolar sealer. However, major study design flaws and limitations in this investigation overshadowed the findings which failed to demonstrate a benefit in using the bipolar sealer over the standard electrocautery method. BS

Criteria for use of omics-based predictors in NCI-sponsored clinical trials.

58 Background: High-throughput omics technologies (e.g., genomics, epigenomics, proteomics, metabolomics) offer exciting opportunities for new biological insights into cancer. The IOM report on translational omics defined omics as the study of related sets of biological molecules in a comprehensive fashion. (IOM (Institute of Medicine) 2012. Evolution of Translational Omics: Lessons Learned and the Path Forward. Washington, DC: The National Academic Press.) The promise of omics technologies has proven problematic to translate into clinically useful tests. Difficulty obtaining biospecimens, unrecognized preanalytical influences, and suboptimal assay analytical performance can lead to unreliable results and conflicting reports. Poor reporting of study details and limited access to data and computer code can thwart efforts to replicate published results or to detect flaws in study design and analysis methods. Methods: NCI held an interactive workshop for a wide variety of stakeholders to explore better approaches to omics-based test development and validation. This workshop heavily informed the ideas presented here. Recommendations are stated concisely, then explained. Results: A checklist of items to consider when evaluating the evidence for clinical use of an omics-based predictor, including in a trial where it will guide therapy, is presented. It covers specimen and assay requirements, the predictor model development process, clinical study design and conduct, and regulatory, ethical, and legal issues. The list applies to any trial involving investigational use of an omics test that will alter the clinical management of patients. The criteria also largely apply to situations in which the test will be evaluated retrospectively on specimens collected from patients who were prospectively enrolled on clinical studies. Conclusions: The proposed checklist should serve as a useful guide to investigators planning to submit proposals for NCI-funded studies involving use of an omics-based test. Ideally, this checklist will be consulted in the assay planning and development phases so that the necessary evidence will have been collected in a well-documented fashion by the time definitive evaluation of the test is desired.

Is primary prevention of Clostridium difficile infection possible with specific probiotics?

The efficacy of probiotics for the prevention of Clostridium difficile infection (CDI) is highly controversial, particularly with regard to the prevention of recurrent CDI. We hypothesize that primary prevention of CDI among patients receiving antibiotics might be a more achievable goal for probiotics than prevention in patients with previous CDI where the host flora is markedly altered. We conducted a literature search for randomized, placebo-controlled efficacy studies of probiotic use among adults receiving antibiotics, in which CDI was one of the outcomes measured. In addition, we conducted meta-analyses of probiotics that were included in more than one randomized trial. Eleven studies were identified; most were seriously underpowered to determine the efficacy of probiotics in the prevention of CDI. Two showed significantly lower rates of CDI among the probiotic recipients. A meta-analysis of three studies that used the probiotic combination Lactobacillus acidophilus CL1285 and Lactobacillus casei LBC80R and a combined analysis of those studies with four studies that used Saccharomyces boulardii, showed lower CDI rates in recipients of probiotics compared with recipients of placebo (risk ratio=0.39; 95% confidence interval 0.19-0.79). While potential flaws in study design were identified, a review of the available literature suggests that the primary prevention of CDI with specific probiotic agents may be achievable. Additional studies of sufficient size and with rigorous design are needed to confirm these findings.

Methodological issues and suggestions for improvement in randomized controlled trials of Chinese herbal medicine for recurrent miscarriage

Recurrent miscarriage is a common disease in clinical obstetrics and gynecology. There is no curative treatment for recurrent miscarriage in conventional medicine. Traditional Chinese medicine (TCM) has been widely used in the treatment of recurrent miscarriage in China for thousands of years. However, randomized controlled trials (RCTs) with high quality evaluating the effectiveness and safety of Chinese herbal medicine (CHM) still lack. This study was conducted in order to identify methodological problems in published or unpublished RCTs of CHM in the treatment of recurrent miscarriage, and provide suggestions for TCM researchers to conduct high-quality clinical research. Literature searching was conducted to collect published and unpublished RCTs from six electronic literature databases, including China National Knowledge Infrastructure, Chongqing VIP Chinese Scientific Journals Database, Chinese Biomedical Literature Database, Wanfang Data, PubMed, and Cochrane Library, till December 15, 2011. The Cochrane Handbook, the CONSORT Statement and the CONSORT for TCM statement were used to assess the methodological quality in the designing and reporting of included trials. A total of 49 RCTs were included in this critical appraisal. Most of the trials were of high risk of bias with flawed study design and poor methodological quality. The main problems included: no report on random methods or sequence concealment; no calculation of sample size; no flow chart of enrollment and outcomes; no report on blinding methods; no or incomplete report on baseline data such as diagnosis criteria and causes of miscarriage recurrent, age, times of spontaneous abortion, TCM syndrome differentiation; complex CHM treatments and inappropriate control interventions; no report on follow-up and withdrawals; inappropriate outcome measurements such as middle-outcome or laboratory test results; no report on final outcome (live-birth rate). There are many methodological problems in RCTs of CHM for recurrent miscarriage. It is suggested that four issues should be taken into consideration for TCM research: basic principles of clinical research are needed in design and reporting of RCTs; definite diagnosis criteria and baseline data should be reported in detail; appropriate treatment and control interventions should be selected according to the assumption and objective of research; final outcome (live-birth rate) should be reported in outcome evaluation.