Poor Methodological Rigor Research Articles

Estimates for diagnostic accuracy of rapid antigen tests for SARS-CoV-2 in systematic reviews are consistently similar despite poor methodological rigor: a methodological overview

ObjectivesTo evaluate the design, conduct, and analysis of systematic reviews on the diagnostic accuracy of rapid antigen tests for SARS-CoV-2 during the COVID-19 pandemic. Study Design and SettingWe did a methodological overview of systematic reviews on diagnostic test accuracy, exploring methodological rigor, risk of bias and potential factors of between-review variability. ResultsOf the 31 included reviews, 30 provided summary statistics for sensitivity and 29 for specificity. Summary sensitivities ranged from 56.2% to 91.1%, with a median of 71.5% (IQR 68.3%–76.6%) and a mean of 72.7% with a 7.2 SD. Reported summary specificity estimates were consistently high: median 99.5% (IQR 99%–99.9%) and a mean of 99.3% with a 0.9 SD. We found methodological shortcomings in the systematic reviews, with a majority showing critically low confidence in quality and a high risk of bias. ConclusionDespite significant methodological flaws in the reviews, the diagnostic accuracy estimates for rapid antigen tests were characterized by a strong central tendency, highlighting the importance of large sample sizes and broad participant representation. This study suggests the need for further research in diagnostic test accuracy assessments of rigor and risk of bias in systematic reviews.

Theories, methodologies, and effects of affect-adaptive games: A systematic review

Affect-adaptive games gained in popularity over the last years in human computer interactions studies, promising potential benefits for player experience, performance, and even health. It is however not yet clear how affective games are being evaluated, what the precise effects are, and how they are based on emotion theoretical concepts that are still not universally agreed upon. This systematic review investigated these questions by analysing relevant high-quality evaluation studies of the effect of affect-adaptive video games on various outcomes in regards to their effects, theoretical assumptions, and methodologies. Out of 3,930 papers, 26 studies were included based on preregistered inclusion and exclusion criteria. A high variance regarding theoretical assumptions and methodological approaches was observed, as well as an overall poor methodological rigour, leading to the conclusion that more work is needed in constructing better methodological standards for game evaluation studies and theoretical considerations when developing and testing affect-adaptive video games.

Ovarian imaging radiomics quality score assessment: an EuSoMII radiomics auditing group initiative

ObjectiveTo evaluate the methodological rigor of radiomics-based studies using noninvasive imaging in ovarian setting.MethodsMultiple medical literature archives (PubMed, Web of Science, and Scopus) were searched to retrieve original studies focused on computed tomography (CT), magnetic resonance imaging (MRI), ultrasound (US), or positron emission tomography (PET) radiomics for ovarian disorders’ assessment. Two researchers in consensus evaluated each investigation using the radiomics quality score (RQS). Subgroup analyses were performed to assess whether the total RQS varied according to first author category, study aim and topic, imaging modality, and journal quartile.ResultsFrom a total of 531 items, 63 investigations were finally included in the analysis. The studies were greatly focused (94%) on the field of oncology, with CT representing the most used imaging technique (41%). Overall, the papers achieved a median total RQS 6 (IQR, −0.5 to 11), corresponding to a percentage of 16.7% of the maximum score (IQR, 0–30.6%). The scoring was low especially due to the lack of prospective design and formal validation of the results. At subgroup analysis, the 4 studies not focused on oncological topic showed significantly lower quality scores than the others.ConclusionsThe overall methodological rigor of radiomics studies in the ovarian field is still not ideal, limiting the reproducibility of results and potential translation to clinical setting. More efforts towards a standardized methodology in the workflow are needed to allow radiomics to become a viable tool for clinical decision-making.Key Points• The 63 included studies using noninvasive imaging for ovarian applications were mostly focused on oncologic topic (94%).• The included investigations achieved a median total RQS 6 (IQR, −0.5 to 11), indicating poor methodological rigor.• The RQS was low especially due to the lack of prospective design and formal validation of the results.

The use of medication adherence guidelines in medication taking behaviour research.

Medication nonadherence continues to be a serious issue in a range of long-term medical conditions and has been studied extensively over the past few decades. However, despite the plethora of research studies on medication adherence, poor methodological rigour in many studies has contributed to limited generalisability of the positive findings, limited impact on patients' medication adherence, and inability to compare between studies. This paper focuses on current guidelines designed specifically for research on medication adherence. It discusses key elements to consider during study design, selection of adherence measurements, and reporting on medication adherence research, to ensure a higher quality of research in medication adherence. Overall, there appears to be variations in adherence terminology reported in the literature despite improvements in defining medication taking behaviour and the availability of taxonomies. In addition, limited guidance exists on how best to measure adherence. Recommendations are provided on appropriate adherence measures for the adherence behaviour being investigated, including careful consideration of adherence concepts, validity of adherence instruments, appropriate instrument selection, definition of nonadherence threshold, and how to report medication adherence. Improving adherence research requires greater clarity and standardisation of descriptions of nonadherence behaviour, increased methodological rigour in study designs, better selection of adherence measurements, and comprehensive reporting.

COVID-19 survey burden for health care workers: literature review and audit

ObjectivesConcerns have been raised about the quantity and quality of research conducted during the COVID-19 pandemic, particularly related to the mental health and wellbeing of health care workers (HCWs). For understanding the volume, source, methodological rigour and degree of overlap in COVID-19, studies were conducted among HCWs in the United Kingdom (UK). Study designMixed methods approach, literature review and audit. MethodsFirst, a literature review of published research studies and second, an audit of studies HCWs have been invited to complete. For the literature review, we searched Medline, PsycINFO and Nexis, webpages of three medical organisations (Royal Society of Medicine, Royal College of Nursing and British Medical Association), and the YouGov website. For the audit, a non-random purposive sample of six HCWs from different London NHS Trusts reviewed email, WhatsApp and SMS messages they received for study invitations. ResultsThe literature review identified 27 studies; the audit identified 70 study invitations. Studies identified by the literature review were largely of poor methodological rigour: only eight studies (30%) provided response rate, one study (4%) reported having ethical approval, and one study (4%) reported funding details. There was substantial overlap in the topics measured. In the audit, volunteers received a median of 12 invitations. The largest number of study invitations were for national surveys (n = 23), followed by local surveys (n = 16) and research surveys (n = 8). ConclusionHCWs have been asked to complete numerous surveys that frequently have methodological shortcomings and overlapping aims. Many studies do not follow scientific good-practice and generate questionable, non-generalisable results.

Systematic review of prospective studies focused on regionalization of care in surgical oncology.

To perform a systematic review of studies prospectively analyzing the impact of regionalization of complex surgical oncology care on patient outcomes. High volume care of complex surgical oncology patients has been repeatedly associated with improved outcomes. Most studies, however, are retrospective and have not prospectively accounted for confounders such as financial ability and social support. Four electronic databases (Ovid MEDLINE®, Ovid EMBASE, Cochrane Library (Wiley), and EBSCHOHost) were searched from inception until August 25, 2018. Two authors independently reviewed 5887 references, with a third independent reviewer acting as arbitrator when needed. Data extracted from 11 articles that met inclusion criteria. Risk of bias assessments conducted using MINORS criteria for the non-randomized, observational studies, and the Cochrane tool for the randomized-controlled trial. Of the 11 studies selected, we found 7 historically-controlled trials, two retrospective cohort studies with prospective data collection, one prospective study, and one randomized-controlled trial. 73% of studies were from Northern Europe, 18% from Ontario, Canada, and 9% from England. Pancreatic surgery accounted for 36% of studies, followed by gynecologic oncology (27%), thoracic surgery (18%), and dermatologic surgery (9%). The studies reported varying outcome parameters, but all showed improvement post-regionalization. Included studies featured poor-to-fair risk of bias. 11 studies indicated improved outcomes following regionalization of surgical oncology, but most exhibit poor methodological rigor. Prospective evidence for the regionalization of surgical oncology is lacking. More research addressing patient access to care and specialist availability is needed to understand the shortcomings of centralization.

An Analysis of Literature on Sport Officiating Research

ABSTRACT Sport officials are crucial members of sport. Researchers have studied their roles numerous times, with results often informing sport procedures (e.g., athlete order in artistic sports). As the research on sport officiating spans five decades and several topics of interest, it is important that researchers periodically synthesize the literature. Purpose: The purpose of this study, therefore, was to conduct an analysis of literature on sport officiating research. Method: Guided by previous researchers, we executed four methodological steps including the article search, article retrieval, sample validity, and article coding. These steps yielded 386 articles for analysis, which ranged from 1971 to 2018. We coded the articles based on four main categories: article information, participant demographics, contextual information, and methodology. Results: Key findings from this analysis include a recent influx in sport officiating research, a vast number of publication journals, few studies dedicated to female-only participants, many studies missing relevant demographic information, an over-representation of interactors, and a reliance on quantitative studies. Conclusions: Though many researchers have conducted studies on sport officiating, several articles had poor methodological rigor (e.g., not reporting key demographic information). In the discussion and conclusion sections, we highlight strengths and weaknesses within the field and provide recommendations to guide future researchers and practitioners, to ensure robust research designs and guide applied practice.

Screening instruments for early detection of autism spectrum disorder in Spanish speaking communities.

Since autism detection protocols rely primarily on parental accounts of early symptoms, the use by Spanish-speaking populations of screening tools developed in a different language and socio-cultural context (usually English) might hamper the success of early detection programs. A systematic search in four databases was completed, identifying 59 tools used for ASD detection. Of these, only nine tools had been applied in Spanish-speaking populations, and of those, only five can be considered specific tools for the early detection of autism. Sensitivity detecting autism was generally lower in the Spanish versions of the reviewed instruments, while specificity tended to be equal to or higher than that reported in the original screeners. Heterogeneity and poor methodological rigor of the studies conducted to date emphasize an urgent need for a concerted effort to develop reliable and valid instruments for the early detection of autism in Spanish-speaking populations worldwide.

Sensationalism and speaking to the public: Scientific rigour and interdisciplinary collaborations in palaeopathology

ObjectivesIn this brief communication we discuss issues concerning scientific rigour in palaeopathological publications, particularly studies published in clinical or general science journals, that employ skeletal analysis to elucidate the lives and deaths of historical figures or interpret “mysterious” assemblages or burials. We highlight the relationship between poor methodological rigour and lack of interdisciplinary communication, and discuss how this can result in scientifically weak, sensational narratives being presented to the public. ConclusionsAlthough most high profile publications involving analysis of archaeological human remains are methodologically sound and well interpreted, others have suffered from poor scientific rigour stemming from an apparent lack of awareness of anthropological methods and ethics. When these publications are highlighted by the press, sensationalistic narratives are perpetuated which may reflect poorly on our discipline and give the public unrealistic expectations about our work. Suggestions for future researchWe suggest that best practice in high-profile paleopathological research include recruitment of a range of authors and reviewers from clinical sciences, anthropology, and the humanities, consideration of the ethical issues surrounding retrospective diagnosis, and transparency with the press in regards to the limitations inherent in this kind of work.

The efficacy of physiotherapy interventions in the treatment of adhesive capsulitis: A systematic review.

Adhesive capsulitis is a debilitating condition which causes the capsule of the gleno-humeral joint to thicken and contract progressively. The effectiveness of various non-operative methods has been demonstrated to improve the pain, range of motion (ROM) and functional status of patients with adhesive capsulitis. This study aims to review recent evidence on the efficacy of physiotherapy interventions in the treatment of adhesive capsulitis. PubMed, Physiotherapy Evidence Database (PEDro), Science Direct and Cochrane databases were searched for studies published since 2013. The search terms included: Frozen shoulder, adhesive capsulitis, physical therapy, rehabilitation, manual therapy, mobilization, exercise, education, and electrotherapy. The search was limited to studies published in English and studies that used human subjects. Quality scores of 33 articles were reviewed according to the Sackett's critical appraisal criteria and the grades of recommendation were determined for physiotherapy interventions used in the studies. The empirical evidence suggests that certain physical therapy techniques and modalities are strongly recommended for pain relief, improvement of ROM, and functional status in patients with adhesive capsulitis, while others are either moderately or mildly recommended. However, the efficacy of one treatment modality over another is uncertain. The poor methodological rigors demonstrated in most of the reviewed studies emphasize the urgent need of properly conducted, adequately sampled randomized controlled trials with adequate follow up to determine the superior combination of treatment.

Methodological Quality of Economic Evaluations in Integrated Care: Evidence from a Systematic Review.

Introduction:The aim of this review is to systematically assess the methodological quality of economic evaluations in integrated care and to identify challenges with conducting such studies.Theory and methods:Searches of grey-literature and scientific papers were performed, from January 2000 to December 2018. A checklist was developed to assess the quality of economic evaluations. Authors’ statements of challenges encountered during their evaluations were qualitatively coded.Results:Forty-four articles were eligible for inclusion. The review found that study design, measurement of cost and outcomes, statistical analysis and presentation of data were the areas with most quality variation. Authors identified challenges mostly related to time horizon of the evaluation, inadequate or lack of comparator group, contamination bias, and a post-hoc evaluation culture.Discussion:Our review found significant differences in quality, with some studies showing poor methodological rigor; challenging conclusions on the cost-effectiveness of integrated care.Conclusion:It is essential for evaluators to use best-practice standards when planning and conducting economic evaluations, in order to build a reliable evidence base for decision-making in integrated care.

Recurrent Oral Ulcers: current and future therapeutic approaches

Recurrent Oral Ulcers: current and future therapeutic approaches

Cross-Cultural Applicability of the Montreal Cognitive Assessment (MoCA): A Systematic Review.

The Montreal Cognitive Assessment (MoCA) is widely used to screen for mild cognitive impairment (MCI). While there are many available versions, the cross-cultural validity of the assessment has not been explored sufficiently. We aimed to interrogate the validity of the MoCA in a cross-cultural context: in differentiating MCI from normal controls (NC); and identifying cut-offs and adjustments for age and education where possible. This review sourced a wide range of studies including case-control studies. In addition, we report findings for differentiating dementias from NC and MCI from dementias, however, these were not considered to be an appropriate use of the MoCA. The subject of the review assumes heterogeneity and therefore meta-analyses was not conducted. Quality ratings, forest plots of validated studies (sensitivity and specificity) with covariates (suggested cut-offs, age, education and country), and summary receiver operating characteristic curve are presented. The results showed a wide range in suggested cutoffs for MCI cross-culturally, with variability in levels of sensitivity and specificity ranging from low to high. Poor methodological rigor appears to have affected reported accuracy and validity of the MoCA. The review highlights the necessity for cross-cultural considerations when using the MoCA, and recognizing it as a screen and not a diagnostic tool. Appropriate cutoffs and point adjustments for education are suggested.

Inconclusive evidence on systemic treatments for recurrent aphthous stomatitis

The trial registers of the Cochrane Oral Health and Pain, Palliative and Supportive Care Groups were searched together with The Cochrane Central database, Medline, Embase, CINHAL and AMED databases. Reference lists of relevant articles were also searched and authors of trials contacted. Randomised controlled trials (RCTs), including those with cross-over design where the primary outcome was a reduction of pain associated with RAS, a reduction in episode duration or episode frequency were included. Data were extracted independently by two authors and risk of bias assessment was conducted on six domains. Cochrane statistical guidelines were to be followed. Twenty-five trials were included; 22 were placebo controlled and eight made head-to-head comparisons (five trials had more than two treatment arms). Twenty-one different interventions were assessed. The interventions were grouped into two categories: immunomodulatory/anti-inflammatory and uncertain. Only one study was assessed as being at low risk of bias. There was insufficient evidence to support or refute the use of any intervention. No single treatment was found to be effective and therefore the results remain inconclusive in regard to the best systemic intervention for RAS. This is likely to reflect the poor methodological rigour of trials, and lack of studies for certain drugs, rather than the true effect of the intervention. It is also recognised that in clinical practice, individual drugs appear to work for individual patients and so the interventions are likely to be complex in nature. In addition, it is acknowledged that systemic interventions are often reserved for those patients who have been unresponsive to topical treatments, and therefore may represent a select group of patients.

Systemic interventions for recurrent aphthous stomatitis (mouth ulcers)

Recurrent aphthous stomatitis (RAS) is the most frequent form of oral ulceration, characterised by recurrent oral mucosal ulceration in an otherwise healthy individual. At its worst RAS can cause significant difficulties in eating and drinking. Treatment is primarily aimed at pain relief and the promotion of healing to reduce the duration of the disease or reduce the rate of recurrence. A variety of topical and systemic therapies have been utilised. To determine the clinical effect of systemic interventions in the reduction of pain associated with RAS, a reduction in episode duration or frequency. We undertook electronic searches of: Cochrane Oral Health Group and PaPaS Trials Registers (to 6 June 2012); CENTRAL via The Cochrane Library (to Issue 4, 2012); MEDLINE via OVID (1950 to 6 June 2012); EMBASE via OVID (1980 to 6 June 2012); CINAHL via EBSCO (1980 to 6 June 2012); and AMED via PubMed (1950 to 6 June 2012). We searched reference lists from relevant articles and contacted the authors of eligible trials to identify further trials and obtain additional information. We included randomised controlled trials (RCTs) in which the primary outcome measures assess a reduction of pain associated with RAS, a reduction in episode duration or a reduction in episode frequency. Trials were not restricted by outcome alone. We also included RCTs of a cross-over design. Two review authors independently extracted data in duplicate. We contacted trial authors for details of randomisation, blindness and withdrawals. We carried out risk of bias assessment on six domains. We followed The Cochrane Collaboration statistical guidelines and risk ratio (RR) values were to be calculated using fixed-effect models (if two or three trials in each meta-analysis) or random-effects models (if four or more trials in each meta-analysis). A total of 25 trials were included, 22 of which were placebo controlled and eight made head-to-head comparisons (five trials had more than two treatment arms). Twenty-one different interventions were assessed. The interventions were grouped into two categories: immunomodulatory/anti-inflammatory and uncertain. Only one study was assessed as being at low risk of bias. There was insufficient evidence to support or refute the use of any intervention. No single treatment was found to be effective and therefore the results remain inconclusive in regard to the best systemic intervention for RAS. This is likely to reflect the poor methodological rigour of trials, and lack of studies for certain drugs, rather than the true effect of the intervention. It is also recognised that in clinical practice, individual drugs appear to work for individual patients and so the interventions are likely to be complex in nature. In addition, it is acknowledged that systemic interventions are often reserved for those patients who have been unresponsive to topical treatments, and therefore may represent a select group of patients.

Infantile colic: A systematic review of medical and conventional therapies

Infantile colic is a prevalent and distressing condition for which there is no proven standard therapy. The aim of this paper is to review medical and conventional treatments for infantile colic. A systematic literature review was undertaken of studies on medical and conventional interventions for infantile colic from 1980 to March 2009. The results and methodological rigour of included studies were analysed using the CONSORT (Consolidated Standards Of Reporting Trials) 2001 statement checklist and Centre for Evidence Based Medicine critical appraisal tools. Nineteen studies and two literature reviews were included for review. Pharmacological studies on Simethicone gave conflicting results and with Dicyclomine hydrochloride and Cimetropium bromide results were favourable but side effects were noted along with issues in study methodology. Some nutritional studies reported favourable results for the use of hydrolysed formulas in bottle-fed infants or low-allergen maternal diets in breastfed infants but not for the use of additional fibre or lactase. There were several issues in regards to methodological rigour. Behavioural studies on the use of increased stimulation gave unfavourable results, whereas results from the use of decreased stimulation and contingent music were favourable. These studies demonstrated poor methodological rigour. There is some scientific evidence to support the use of a casein hydrolysate formula in formula-fed infants or a low-allergen maternal diet in breastfed infants with infantile colic. However, there is little scientific evidence to support the use of Simethicone, Dicyclomine hydrochloride, Cimetropium bromide, lactase, additional fibre or behavioural interventions. Further research of good methodological quality on low-allergenic formulas and maternal diets is indicated.

Commentary on Oh H and Seo W (2003) A meta‐analysis of the effects of various interventions in preventing endotracheal suction‐induced hypoxemia. Journal of Clinical Nursing 12, 912–924

Commentary on Oh H and Seo W (2003) A meta‐analysis of the effects of various interventions in preventing endotracheal suction‐induced hypoxemia. <i>Journal of Clinical Nursing</i> 12, 912–924

Interventions with family caregivers for people with dementia: efficacy problems and potential solution.

Family members are increasingly caring for relatives with dementia at home. It is well recognized that this is a stressful experience for carers. Treatment approaches that aim to alleviate this stress were introduced in the 1970s and later research began to evaluate their efficacy. Early studies using subjective ratings of satisfaction and usefulness demonstrated encouraging results. However, later studies using standardized measures of distress and burden, have not been able to demonstrate efficacy in a convincing way. There have been some positive significant changes; however, most findings have been poor, inconsistent and equivocal. This may be accounted for by the poor methodological rigour diluting the positive potential benefits of interventions for carers. This paper suggests that in order for the efficacy research to evolve, there needs to be a change in direction. It is vital that instead of repeating 'more of the same' studies, researchers need to prioritize the use of theoretically driven interventions and research design. This factor alone could provide the framework to influence the methodological issues that potentially dilute the demonstration of treatment efficacy. Without clearer evidence from research, guidance for practitioners about treatment for carers is ambiguous.