Unweighted Kappa Coefficient Research Articles

Metastatic Patterns: Insights from Japanese Pathological Autopsy Registry Analysis.

Understanding the metastatic patterns is crucial for the treatment of malignancies. This study aimed to identify the characteristic organ metastases of primary malignancies, including rare malignancies, and classify them according to their metastatic patterns. We extracted data on primary malignancies and organ metastases from the Annual of Pathological Autopsy Cases in Japan recorded in 1993-2021. Autopsy findings of the primary and metastatic organs in patients with malignancy were recorded on an organ-by-organ basis. The metastatic frequency (number of metastases per autopsy) and the proportion (percentage of organs with metastases out of the total in a primary malignancy) for 48 organ metastasis sites across 76 primary malignancies were calculated. Metastatic patterns were classified into hierarchical and non-hierarchical clustering classifications based on the standard proportion of organ metastases. A total of 332,195 autopsy cases and 810,206 organ metastases were analyzed. The metastatic frequency of all malignancies was 2.44. Malignancies of the placenta, eye, and ovary showed a higher propensity for metastasis, whereas central nervous system malignancies showed a lower tendency. Metastasis site was a characteristic of each malignancy, with a particularly high proportion of lung metastasis in parathyroid malignancy and bone metastasis in prostate malignancy. In the hierarchical and non-hierarchical cluster methods, brain, lung, liver, bone, peritoneum, and hematolymphoid organ were key metastatic sites, and this factor divided primary malignancies into seven categories. The unweighted kappa coefficient comparing the two classification methods was 0.84 (95% confidence interval: 0.75-0.93). The proportion of metastatic organs was influenced by anatomical location and/or organ specificity of the primary malignancies. Our study provides a comprehensive overview of the patterns and frequencies of metastatic organ sites associated with 76 primary malignancies. Our findings will provide useful information for future research and clinical practice.

Level of Agreement in Subjective Selection of Gingival Colour

Background and Objectives: Primary outcome: To assess the level of agreement between the objective and subjective methods for recording gingival colour in each area of the gingiva. Secondary outcome: To compare performance of the subjective visual method of gingival colour selection by a male observer and a female observer. Materials and Methods: A chromatic study was conducted on a total of 101 participants, in five gingival zones, from the free gingival margin to the mucogingival line, using a SpectroShade Micro spectrophotometer for the objective method and 21 ad hoc ceramic gingival shade tabs for the subjective method. A man and a woman of the same age, with the same amount of clinical experience in dentistry, selected the tab that most resembled the colour of participants’ gingiva. The “chromatic error” was then assessed by calculating the colour difference (using the Euclidean and CIEDE2000 formulae) between the CIELAB coordinates of the shade tab selected and the objective coordinates of the gingiva. The unweighted Kappa coefficient was used to calculate the level of agreement between observers. Results: For the male observer, the mean chromatic errors varied between ΔEab* 10.3 and 13.1 units, while for the female observer, the mean errors varied between ΔEab* 11.1 and 12.8: these differences were not statistically significant. Similarly, no statistically significant differences were found between the mean chromatic errors for the five gingival zones in either the male operator (p = 0.100) or the female operator (p = 0.093). The minimum level of agreement (unweighted Kappa) between the observers ranged from 0.1 to 0.4. Conclusions: Subjective selection of gingival colour was very inaccurate, by both the male observer and the female observer, for any area of the gingiva, with no differences identified between them. The level of agreement between the observers was low. These findings suggest that gingival colour should not be determined using solely subjective methods, given that the chromatic errors significantly exceeded the clinical acceptability threshold for gingiva (4.1 units for ΔEab* and 2.9 units for ΔE00). Both observers showed a tendency to select gingival shade tabs that were redder and bluer than the objective colours.

Comparison of physician- and self-assessed pubertal onset in Japanese children.

Physical examinations to assess pubertal development are challenging in large epidemiological surveys. This study aimed to assess the reliability of judgment of pubertal onset in Japanese children by the original pubertal self-assessment sheet. A total of 144 children aged 10 or 12 years were recruited between March 2019 and September 2020 from the pediatric endocrine outpatient clinics of participating institutions. Agreement between the physician- and participantassessed pubertal onsets was determined using unweighted kappa (UK) and Gwet's agreement coefficient (AC1). The physician's assessment of pubertal onset was in slight agreement with that of the self-assessment sheet in 10-year-old boys (UK: 0.23 and AC1: 0.14), whereas the agreement between the physician's assessment and self-assessment sheet results was good and the physician's assessment was fair (UK: 0.64 and AC1: 0.94) in 12-year-old boys. The physician's assessment of pubertal onset were in good and moderate agreement with the self-assessment sheet in 10-year-old girls (UK/AC1: 0.74/0.78, respectively). In 12-year-old girls, although it showed poor agreement with UK (0.46), there was a very good agreement with AC1 (0.88). Although self-assessment of breast development was in good agreement with that of the physician's assessment for determining pubertal onset in girls, large-scale epidemiological studies are difficult to conduct for adolescent boys, especially for those in the early pubertal stage.

Intra- and inter-observer concordance of a fascial standing flexion test in children

IntroductionManual clinical tests must be highly reliable and valid to help in diagnosing, monitoring, and treating patients. Recent research highlighted the role of the fascia; thus, it is important to develop inexpensive, easy-to-use, and clinically valid procedures to assess this tissue in the field of manual therapy. The purpose of this study was to determine the level of inter- and intra-examiner concordance when performing a fascial standing flexion test in children. DesignThe present study is a descriptive observational study. MethodsTwo examiners tested 24 healthy children between 11 and 12 years of age performing two trials with the proposed fascial test. Inter- and intra-examiner data were analyzed using unweighted kappa coefficients and percentage agreement. ResultsWe observed zero inter-examiner reliability in the fascial standing flexion test (mean κ = −0.071, mean percentage agreement of 43.7%), and moderate intra-examiner reliability (mean κ = 0.693, mean percentage agreement of 85.4%). ConclusionThe reliability and validity of the fascial standing flexion test must be improved before it can be recommended for use in a clinical setting.

Inter-rater reliability of the Australian Spasticity Assessment Scale in poststroke spasticity.

To investigate the inter-rater reliability of the Australian Spasticity Assessment Scale (ASAS) in adult stroke patients with spasticity, two experienced clinicians rated the elbow flexor, wrist flexor, and ankle plantar flexor spasticity by using the ASAS in 85 persons with stroke. Unweighted and weighted (linear and quadratic) kappa statistics were used to calculate the inter-rater reliability for each muscle group. Unweighted kappa coefficients for elbow flexors (n = 83), wrist flexors (n = 80), and ankle plantar flexors (n = 77) were 0.67, 0.60, and 0.55, respectively. Linear and quadratic weighted kappa coefficients, respectively, were 0.77 and 0.87 for elbow flexors, 0.72 and 0.82 for wrist flexors, and 0.72 and 0.85 for ankle plantar flexors. The raters never disagreed by more than a single score in the rating of elbow flexors. On the contrary, the raters disagreed by more than a single score in three patients in the rating of ankle plantar flexors and in one patient in the rating of wrist flexors. The results suggested that inter-rater reliability of the ASAS differed according to the spastic muscle group assessed and the statistical method used. The strength of the agreement on the ASAS, an ordinal scale, ranged from good to very good when the weighted kappa values were considered.

Light-based diagnostic methods for the in vivo assessment of initial caries lesions: Laser fluorescence, QLF and OCT.

To evaluate different light-based caries diagnostic methods for assessment of non-cavitated initial carious lesions on smooth surfaces. 39 participants were included. For each participant, three regions of interest (ROI) with ICDAS codes 0, 1 or 2 were defined. All ROIs (n = 117) were investigated/imaged with laser fluorescence (LF, DIAGNOdent 2095; KaVo dental GmbH), quantitative light-induced fluorescence (QLF; Biluminator 2+, Inspektor Research Systems B.V.) and spectral-domain optical coherence tomography (SD-OCT; Telesto II, Thorlabs GmbH). The values of LF, ΔF of QLF, and the lesion extent assessed by OCT were categorized. Frequency distribution of LF-/QLF- and OCT-scores was determined for each ICDAS code included. Inter- and intra-examiner reproducibility of QLF and OCT measurements were assessed by unweighted kappa coefficient (ĸ) and Wilcoxon test (α = 0.05). Compared with LF and QLF, OCT showed various manifestations of carious lesions for visually sound ROIs and a larger variation of caries extent in depth within the same ICDAS code groups. Intra-examiner reproducibility ranged between 0.49 and 1.00 for the QLF analysis and between 0.95 and 0.99 for the OCT analysis. Inter-examiner reproducibility ranged between 0.17 and 0.32 for the QLF analysis and between 0.65 and 0.79 for the OCT analysis. LF and QLF were insensitive to less pronounced smooth-surface lesions while OCT allowed differentiation based on the penetration depth of the carious lesions. This makes OCT a suitable method to complement conventional visual inspection in order to detect and assess (very) early lesions.

Evaluating sensitivity and specificity of handheld point-of-care ultrasound testing for gynecologic pathology: a pilot study for use in low resource settings

BackgroundPoint-of-care ultrasound (POC-US) is a diagnostic test conducted at the site of patient care with direct interpretation by the clinician, providing immediate results. POC-US for gynecologic application is not well characterized by current literature yet has the potential to increase access in limited resource settings. We compared the diagnostics of three POC-US devices for gynecologic (GYN) pathology and then performed evaluation of sensitivity and specificity of a single best POC-US device for intended use in a low resource setting.MethodsThis is prospective, pilot descriptive study of 60 subjects. In part 1, comparison of three POC-US devices was performed. Twenty subjects underwent POC-US with three test units [GE Vscan (Vscan), Sonosite Iviz (Iviz), Philips Lumify (Lumify)] followed by diagnostic ultrasound (Dx-US) for reference imaging. Image quality and correlation for devices was scored by blinded reviewers and quantitative measurements of GYN pathology were compared. In part 2, forty subjects underwent POC-US validation with the highest scoring device (Lumify) and Dx-US for reference imaging. Concordance of POC-US operator-interpreted diagnosis with reference imaging interpretation were assessed by Cohen’s unweighted kappa coefficient. Accuracy and agreement of POC-US were assessed by linear regression and Bland–Altman plot analysis. Sensitivity and specificity of POC-US for gynecologic pathologies were calculated.ResultsIn aggregate qualitative measurements, Lumify and Iviz units performed superiorly to Vscan. There was no statistically significant difference in quantitative measurements between devices, but a trend towards lower mean error was seen for Lumify and Iviz as compared to Vscan. Lumify device had highest overall scoring and was selected for further testing. In validation comparison of Lumify to Dx-US, no statistically significant differences were found for measurements of endometrium, uterus, ovaries, adnexal pathology, or leiomyomata, (P < 0.02) with excellent agreement in operator-interpreted diagnosis (Kappa > 0.7). Sensitivity and specificity of detecting pathology was 80–100% with PPV and NPV 76–100%.ConclusionAmong three POC-US devices, Lumify and Iviz devices show highest potential for successful application to clinical gynecologic ultrasound. Clinician-performed POC-US has high diagnostic accuracy, sensitivity, and specificity for basic GYN anatomy and pathology. POC-US is an acceptable and feasible diagnostic tool with potential for future application in a low resource setting to increase access to ultrasound.

Assessment of Caries Spectrum of First Permanent Molars in 7- to 8-Year-Old School Children in Northern Saudi Arabia: A Cross-Sectional Study

Objective: To estimate caries spectrum in first permanent molars (FPMs) of 7- to 8-year-old children in Saudi Arabia using Caries Assessment Spectrum and Treatment (CAST) index, and examine the correlation between distributions of CAST codes among contralateral and opposite arches. Material and Methods: A cross-sectional survey was conducted of 7- to 8-year-old Saudi school children in Sakaka, Aljouf, Saudi Arabia. A total of 385 children were examined. CAST codes were determined for FPMs, and their distribution was analyzed using Spearman’s rank correlation coefficient. Statistical significance was set as p<0.05. Intra-examiner reliability was determined using the unweighted kappa coefficient. Results: The results indicated most of the FPMs were scored as code 3 (enamel lesions) such as upper contralateral FPM 44.40% for right, 53.50% for left, respectively, and lower contralateral molars were 61.30% for right and 64.90% for left, respectively. It was also observed that upper FPMs had scored code 0 (sound teeth) more than that of lower FPMs. Serious morbidity (codes 6 and 7) and mortality (code 8) showed a low prevalence in FPMs. Strong correlations were observed between upper contralateral FPMs (r=0.586) and lower contralateral FPMs (r=0.567, p<0.001). Conclusion: A high prevalence of enamel caries lesions (code 3) was observed in FPMs; in addition, a strong correlation was present between upper contralateral FPMs and between lower contralateral FPMs.

Patients with cystic echinococcosis in the three national referral centers of Mongolia: A model for CE management assessment.

BackgroundMongolia is one of the endemic countries for cystic echinococcosis (CE), a zoonotic disease caused by the larval stage of Echinococcus granulosus. The goal of this study is to describe the current clinical management of CE in Mongolia, to capture the distribution of cyst stages of patients treated, and to contrast current practice with WHO-IWGE expert consensus.MethodsHospital records of CE patients treated between 2008 and 2015 at the three state hospitals and fulfilling the inclusion criterion ‘discharge diagnosis CE’ (ICD 10 code B.67.0–67.9) were reviewed. Demographical, geographical, clinical and ultrasonography (US) data were extracted and analyzed. The annual surgical incidence was estimated. The digital copies of US cyst images were independently staged by three international experts following the WHO CE cyst classification to determine the proportions of patients which ideally would have been assigned to the WHO recommended treatment modalities surgery, percutaneous, medical (benzimidazole) treatment and watch & wait.ResultsA total of 290 patient records fulfilled the inclusion criteria of the study. 45.7% of patients were below 15 years of age. 73.7% of CE cysts were located in abdominal organs, predominantly liver. US images of 84 patients were staged and assessed for interrater-agreement. The average raw agreement was 77.2%. Unweighted Kappa coefficient and weighted Kappa was 0.57 and 0.59, respectively. Mean proportions of images judged as stages CE1, CE2, CE3a, CE3b, CE4 and CL were 0.59, 0.01, 0.19, 0.08, 0.03 and 0.11, respectively. 40 cysts met the inclusion criteria of treatment modality analysis. The mean proportions of cases with a single cyst assigned to medical, percutaneous treatment, surgery and watch & wait were 52.5% (95% CI 42–65), 25.8% (95% CI 15–30), 5.1% (95% CI 0–10) and 3.3% (95% CI 0–10), respectively. 13.3% (95% CI 5–25) of cysts were staged as CL and therefore assigned to further diagnostic requirement.ConclusionWHO CE cyst classification and WHO-IWGE expert consensus on clinical CE management is not implemented in Mongolia. This results in exclusively surgical treatment, an unnecessary high risk approach for the majority of patients who could receive medical, percutaneous treatment or observation (watch & wait). Introduction of WHO-IWGE expert consensus and training in ultrasound CE cyst staging would be highly beneficial for patients and the health care services.

Extended field imaging using swept-source optical coherence tomography angiography in retinal vein occlusion.

To evaluate the degree of ischemia in eyes with retinal vein occlusion (RVO) using swept-source optical coherence tomography angiography (SS-OCTA) with the extended field imaging (EFI) technique, which extends the area encompassed by SS-OCTA by scanning through trial frames fitted with a +20-diopter lens. Retrospective observational study. Twenty-three consecutive eyes of 22 patients with RVO underwent 12 × 12mm SS-OCTA imaging both with and without EFI for determination of extension rate. Two graders blinded to the clinical data evaluated the degree of retinal ischemia in paired EFI-SS-OCTA and fluorescein angiography (FA) images, and the concordance rates between the grades were statistically examined. One EFI-SS-OCTA image was not successfully obtained due to motion artifacts caused by the patient's poor central vision, while SS-OCTA images without EFI were captured in all 23 eyes. The average extension rate of EFI-SS-OCTA over SS-OCTA was 1.39 ± 0.06 and the average scanning area was enlarged by 76.4%. Two graders evaluated the degree of retinal ischemia by measuring nonperfusion areas as the sum of disc areas/diameters. Although their assessments of the EFI-SS-OCTA images were in complete agreement (Cohen's Unweighted Kappa coefficient = 1.00), concordance using FA images was only moderate (Cohen's Unweighted Kappa coefficient = 0.60). EFI-SS-OCTA noninvasively produces wider field images of retinal vasculature with one capture and provides resolution sufficient to accurately evaluate retinal capillary nonperfusion in RVO.

Oral medicine acceptance in infants and toddlers: measurement properties of the caregiver-administered Children\u2019s acceptance tool (CareCAT)

BackgroundDeveloping age-appropriate medications remains a challenge in particular for the population of infants and toddlers, as they are not able to reliably self-report if they would accept and consequently take an oral medicine. Therefore, it is common to use caregivers as proxies when assessing medicine acceptance. The outcome measures used in this research field differ and most importantly lack validation, implying a persisting gap in knowledge and controversy in the field. The newly developed Caregiver-administered Children’s Acceptance Tool (CareCAT) is based on a 5-point nominal scale, with descriptors of medication acceptance behavior. This cross-sectional study assessed the measurement properties of the tool with regards to the user’s understanding and its intra- and inter-rater reliability.MethodsParticipating caregivers were enrolled at a primary healthcare facility where their children (median age 6 months) had been prescribed oral antibiotics. Caregivers, trained observers and the tool developer observed and scored on the CareCAT tool what behavior children exhibited when receiving the medicine (n = 104). The video-records of this process served as replicate observations (n = 69). After using the tool caregivers were asked to explain their observations and the tool descriptors in their own words. The tool’s reliability was assessed by percentage agreement and Cohen’s unweighted kappa coefficients of agreement for nominal scales.ResultsThe study found that caregivers using CareCAT had a satisfactory understanding of the tool’s descriptors. Using its dichotomized scores the tool reliably was strong for acceptance behavior (agreement inter-rater 84–88%, kappa 0.66–0.76; intra-rater 87–89%, kappa 0.68–0.72) and completeness of medicine ingestion (agreement inter-rater 82–86%, kappa 0.59–0.67; intra-rater 85–93%, kappa 0.50–0.70).ConclusionsThe CareCAT is a low-cost, easy-to-use and reliable instrument, which is relevant to assess acceptance behavior and completeness of medicine ingestion, both of which are of significant importance for developing age-appropriate medications in infants and toddlers.

A new SPECT/CT reconstruction algorithm: reliability and accuracy in clinical routine for non-oncologic bone diseases

BackgroundxSPECT Bone® (xB) is a new reconstruction algorithm developed by Siemens® in bone hybrid imaging (SPECT/CT). A CT-based tissue segmentation is incorporated into SPECT reconstruction to provide SPECT images with bone anatomy appearance. The objectives of this study were to assess xB/CT reconstruction diagnostic reliability and accuracy in comparison with Flash 3D® (F3D)/CT in clinical routine. Two hundred thirteen consecutive patients referred to the Brest Nuclear Medicine Department for non-oncological bone diseases were evaluated retrospectively. Two hundred seven SPECT/CT were included. All SPECT/CT were independently interpreted by two nuclear medicine physicians (a junior and a senior expert) with xB/CT then with F3D/CT three months later. Inter-observer agreement (IOA) and diagnostic confidence were determined using McNemar test, and unweighted Kappa coefficient. The study objectives were then re-assessed for validation through > 18 months of clinical and paraclinical follow-up.ResultsNo statistically significant differences between IOA xB and IOA F3D were found (p = 0.532). Agreement for xB after categorical classification of the diagnoses was high (κxB = 0.89 [95% CI 0.84 –0.93]) but without statistically significant difference F3D (κ F3D = 0.90 [95% CI 0.86 – 0.94]). Thirty-one (14.9%) inter-reconstruction diagnostic discrepancies were observed of which 21 (10.1%) were classified as major. The follow-up confirmed the diagnosis of F3D in 10 cases, xB in 6 cases and was non-contributory in 5 cases.ConclusionsxB reconstruction algorithm was found reliable, providing high interobserver agreement and similar diagnostic confidence to F3D reconstruction in clinical routine.

Outcome stability of orthopaedic treatment of Skeletal Class III Malocclusion: A study of prognostic agreement between three models

Introduction: Long-term stability of the treatment outcome of skeletal Class III malocclusion is always not achievable, therefore several prediction models of stability of orthopaedic treatment of the Class III malocclusion have been proposed.Aim: This cross-sectional study was aimed at the evaluation of the prognostic agreement and association between three different prediction models based on skeletal parameters recorded on lateral cephalograms.Materials and methods: A total of 75 subjects (34 females and 41 males, age range 7-11, mean age 9.1±1.2 years) having skeletal ClassIII malocclusion were included in the study. Prediction models reported by Baccetti et al. 2004 (prediction model 1), Moon et al. 2005 (prediction model 2) and Yoshida et al. 2006 (prediction model 3), were considered. Percentage agreement and unweighted kappa coefficient evaluated the agreement between the prediction models in terms of stable or unstable cases. Multiple regressions were run to evaluate the association between the absolute scores obtained from each of the prediction model (prediction scores).Results: Percentage agreement ranged from 77.3% (models 2 and 3) to 89.3%, (models 1 and 2) and corresponding unweighted kappa coefficients ranged from 0.099 (models 1 and 3) to 0.205 (models 2 and 3). The prediction score from model 3 was significantly associated with those from both the other models; on the contrary, prediction scores from models 1 and 2 were not significantly associated.Conclusions: Agreement in terms of stability or instability outcome derived from these models is only partial and not satisfactory,even though significant association between the prediction scores denotes a common biological meaning of the models.

The new SPECT/CT reconstruction algorithm xSPECT Bone®: Reliability and accuracy in daily practice for non-oncologic bone diseases

Background xSPECT Bone® (xB) is a new reconstruction algorithm developed by Siemens® in bone hybrid imaging (SPECT/CT). A CT-based tissue segmentation is incorporated into SPECT reconstruction to provide SPECT images with bone anatomy appearance. The objectives of this unique study were, in a context of daily practice, to compare inter-observer agreement (IOA) and diagnostic accuracy between two Siemens® reconstructions (a new one xB/CT and the commonly used Flash 3D ® (F3D)/CT). Methods Two hundred thirteen consecutive patients referred to the Brest nuclear medicine department for non-oncological bone diseases were evaluated retrospectively. Each SPECT/CT was performed on a Symbia Intevo T6 gamma camera (Siemens®). Two hundred seven SPECT/CT were included (7 patients were excluded and 2 SPECT/CT were performed for the same patient). All SPECT/CT were independently interpreted by two nuclear physicians (one junior and one senior expert) with xB/CT then with F3D/CT three months later. The diagnoses were classified into 5 categories (Normal scintigraphy; Arthritis; Abarticular disease; Fracture or tumour pathology; Complex Regional Pain Syndrome). IOA and diagnostic accuracy were determined using a McNemar test, an unweighted Kappa coefficient and using the clinical and paraclinical follow-up of more than 18 months. Results No statistically significant difference between IOA xB and IOA F3D were found (P = 0.532). Agreement for xB after categorical classification of the diagnoses was high (κ xB = 0.89 [95% CI 0.84–0.93]) but without statistically significant difference with that of F3D (κ F3D = 0.90 [95% CI 0.86–0.94]). Thirty-one (14.9%) inter-reconstruction diagnostic discrepancies were observed whose 21 (10.1%) were classified as major. The follow-up confirmed the diagnosis of F3D in 10 cases, xB in 6 cases and was non-contributory in 5 cases. Conclusions xB reconstruction algorithm was found reliable, providing high inter-observer agreement with similar performances to F3D reconstruction in a context of daily practice. Nevertheless, we believe it provides a strong added value to the SPECT/CT images thanks to a striking aesthetic aspect, with the condition of a strict spatial agreement between the SPECT and the CT acquisitions to obtain interpretable xB images.

Reliability of the radiographic variables in the International Spinal Cord Injury Spinal Column Injury Basic Data Set compared with the AO classification.

Intra- and interrater reliability study for radiological variables of the International Spinal Cord Injury (SCI) Spinal Column Injury Basic Data Set. To test reliability of the radiological variables in the International SCI Spinal Column Injury Basic Data Set and compare it with the Arbeitsgemeinschaft für Osteosynthesefragen (AO) classification. The database of Eastern Denmark Regional SCI Referral Center, Copenhagen, Denmark. Ratings of the International SCI Spinal Column Injury Basic Data Set radiological variables and AO classification were obtained by two international observers for all the surgically treated spine trauma patients between 1st October 2010 and 31st December 2012 at the Spine Unit, Rigshospitalet, Denmark. Statistical analyses for intra- and interrater crude agreement and Cohen's unweighted kappa (κ) coefficients were performed. For 283 spine injuries, the intra- and interrater reliability for the individual radiological variables of the International SCI Spinal Column Injury Basic Data Set was at least substantial (κ=0.67-0.97 for interrater, κ=0.79-0.89 for the intrarater agreement). For the AO classification, intrarater reliability was moderate-to-substantial (κ=0.57-0.75), whereas interrater reliability was substantial (κ=0.67-0.69). The crude intra- and interrater agreement for a combined radiographic SCI Spinal Column Injury Basic Data Set variable showed no significant difference compared with the AO classification (P=0.067-0.895). The reliability of International SCI Spinal Column Injury Basic Data Set radiological variables is comparable to the AO classification system. We encourage its use for spinal column injury description, thus facilitating data collection and comparison between centres and countries.

Development, inter-rater reliability and feasibility of a checklist to assess implementation (Ch-IMP) in systematic reviews: the case of provider-based prevention and treatment programs targeting children and youth.

BackgroundSeveral papers report deficiencies in the reporting of information about the implementation of interventions in clinical trials. Information about implementation is also required in systematic reviews of complex interventions to facilitate the translation and uptake of evidence of provider-based prevention and treatment programs. To capture whether and how implementation is assessed within systematic effectiveness reviews, we developed a checklist for implementation (Ch-IMP) and piloted it in a cohort of reviews on provider-based prevention and treatment interventions for children and young people. This paper reports on the inter-rater reliability, feasibility and reasons for discrepant ratings.MethodsChecklist domains were informed by a framework for program theory; items within domains were generated from a literature review. The checklist was pilot-tested on a cohort of 27 effectiveness reviews targeting children and youth. Two raters independently extracted information on 47 items. Inter-rater reliability was evaluated using percentage agreement and unweighted kappa coefficients. Reasons for discrepant ratings were content analysed.ResultsKappa coefficients ranged from 0.37 to 1.00 and were not influenced by one-sided bias. Most kappa values were classified as excellent (n = 20) or good (n = 17) with a few items categorised as fair (n = 7) or poor (n = 1). Prevalence-adjusted kappa coefficients indicate good or excellent agreement for all but one item. Four areas contributed to scoring discrepancies: 1) clarity or sufficiency of information provided in the review; 2) information missed in the review; 3) issues encountered with the tool; and 4) issues encountered at the review level. Use of the tool demands time investment and it requires adjustment to improve its feasibility for wider use.ConclusionsThe case of provider-based prevention and treatment interventions showed relevancy in developing and piloting the Ch-IMP as a useful tool for assessing the extent to which systematic reviews assess the quality of implementation. The checklist could be used by authors and editors to improve the quality of systematic reviews, and shows promise as a pedagogical tool to facilitate the extraction and reporting of implementation characteristics.Electronic supplementary materialThe online version of this article (doi:10.1186/s12874-015-0037-7) contains supplementary material, which is available to authorized users.

The Verbal Rating Scale Is Reliable for Assessment of Postoperative Pain in Hip Fracture Patients.

Background. Hip fracture patients represent a challenge to pain rating due to the high prevalence of cognitive impairment. Methods. Patients prospectively rated pain on the VRS. Furthermore, patients described the changes in pain after raising their leg, with one of five descriptors. Agreement between paired measures on the VRS at rest and by passive straight leg raise with a one-minute interval between ratings at rest and three-minute interval for straight leg raise was expressed by kappa coefficients. Reliability of this assessment of pain using the VRS was compared to the validity of assessing possible change in pain from the selected descriptors. Cognitive status was quantified by the short Orientation-Memory-Concentration Test. Results. 110 patients were included. Paired scores with maximum disagreement of one scale point reached 97% at rest and 95% at straight leg raise. Linear weighted kappa coefficients ranged from 0.68 (95% CI = 0.59–0.77) at leg raise to 0.75 (95% CI = 0.65–0.85) at rest. Unweighted kappa coefficients of agreement in recalled pain compared to agreement of paired VRS scores ranged from 0.57 (95% CI = 0.49–0.65) to 0.36 (95% CI = 0.31–0.41). Interpretation. The VRS is reliable for assessment of pain after hip fracture. The validity of intermittent questioning about possible change in pain intensity is poor.

Manual Ability Classification System (MACS): reliability between therapists and parents in Brazil.

BACKGROUND:The Manual Ability Classification System (MACS) has been widely used to describe the manual ability of children with cerebral palsy (CP); however its reliability has not been verified in Brazil. OBJECTIVE:To establish the inter- and intra-rater reliability of the Portuguese-Brazil version of the MACS by comparing the classifications given by therapists and parents of children with CP. METHOD:Data were obtained from 90 children with CP between the ages of 4 and 18 years, who were treated at the neurology and rehabilitation clinics of a Brazilian hospital. Therapists (an occupational therapist and a student) classified manual ability (MACS) through direct observation and information provided by parents. Therapists and parents used the Portuguese-Brazil version of the MACS. Intra- and inter-rater reliability was obtained using unweighted Kappa coefficient (k) and intra-class correlation coefficient (ICC). The Chi-square test was used to identify the predominance of disagreements in the classification of parents and therapists. RESULTS:An almost perfect agreement resulted among therapists [K=0.90 (95% CI 0.83-0.97); ICC=0.97 (95%CI 0.96-0.98)], as well as with intra-rater (therapists), with Kappa ranging between 0.83 and 0.95 and ICC between 0.96 and 0.99 for the evaluator with more and less experience in rehabilitation, respectively. The agreement between therapists and parents was fair [K=0.36 (95% CI 0.22-0.50); ICC=0.79 (95% CI 0.70-0.86)]. CONCLUSIONS:The Portuguese version of the MACS is a reliable instrument to be used jointly by parents and therapists.

Dental caries in primary and permanent molars in 7-8-year-old schoolchildren evaluated with Caries Assessment Spectrum and Treatment (CAST) index.

BackgroundNo reports on a caries pattern covering the full spectrum of the disease could be found in the literature. The aim of this study was to evaluate caries in primary and first permanent molars of 7-8-year-old Polish children by the Caries Assessment Spectrum and Treatment (CAST) index and to find whether there was any correlation between the caries stages in such teeth.MethodsThe study covered 284 7-8-year-old children from randomly selected schools in the Bialystok District, Poland. The prevalence of CAST categories was evaluated with regard to the first and second primary, and first permanent, molars. The Spearman’s rank correlation coefficient was used to explore the correlation of the distribution of CAST codes among the evaluated teeth. The level of statistical significance was established at p < 0.05. The intra-examiner reliability was determined by the unweighted kappa coefficient.ResultsWith regard to the permanent molars, caries was observed in 14.8% to 17.3% of the molar and most lesions were scored at the non-cavitation level. Caries in primary molars was most often recorded at the stage of cavitated dentine lesion. Teeth with pulpal involvement, sepsis and extracted due to caries were found to be more prevalent in first, and then in second primary molars. A strong correlation was found between the status of teeth from the right and left sides of the oral cavity. The correlation of the status of first and second primary teeth was stronger for the left than for the right side of the mouth, r was 0.627 and 0.472 in maxilla and 0.513 and 0.483 in mandible (p < 0.001), respectively. For the neighbouring primary and permanent molars the correlation was assessed to be weak. With regard to the teeth situated in opposite jaws the study revealed that the correlations were moderate - r between 0.33 and 0.49. The intra-examiner reliability was established at 0.96 for the primary dentition and at 0.878 for permanent molars.ConclusionThe strongest correlation found in the evaluated population concerned the distribution of caries in primary molars on the left side of the mouth. The study proved the usefulness of the CAST index in epidemiological surveys.

Intra- and Interobserver Reproducibility of Grading ofHelicobacter pyloriGastritis According to the Updated Sydney System

Aims. To study the intra- and interobserver reproducibility of Sydney System amongst pathologists in Cameroon as Sydney System gradation has not gained enough confidence in African pathologists. Methods. We performed a descriptive study including 100 patients who benefited from gastric biopsy by endoscopy. These biopsy specimens were stained with hematein and eosin and modified Giemsa, and read independently using the same microscope by two pathologists with four years experience and no experience with the updated Sydney System. Gastritis was graded according to the updated Sydney System. Levels of intra- and interobserver reproducibility were assessed using the unweighted kappa coefficient. Results. The intraobserver reproducibility of gradation of Helicobacter pylori density, activity, chronic inflammation, atrophy, and intestinal metaplasia showed respective values of kappa: 0.63, 0.34, 0.61, 0.48, and 0.82 for one observer against 0.42, 0.005, 0.41, 0.31, and 0.72 for the other. Interobserver reproducibility kappa values were, respectively, 0.41, 0.18, 0.57, 0.58, and 0.82. Conclusion. Results are encouraging but experience in the updated Sydney System should be improved. The later should be introduced as a means to grade and classify gastritis in Cameroon and African countries.