Single Intraclass Correlation Research Articles

Diffusion tensor imaging within the healthy cervical spinal cord: Within- participants reliability and measurement error

BackgroundDiffusion tensor imaging (DTI) is a promising technique for the visualization of the cervical spinal cord (CSC) in vivo. It provides information about the tissue structure of axonal white matter, and it is thought to be more sensitive than other MR imaging techniques for the evaluation of damage to tracts in the spinal cord. AimThe purpose of this study was to determine the within-participants reliability and error magnitude of measurements of DTI metrics in healthy human CSC. MethodsA total of twenty healthy controls (10 male, mean age: 33.9 ± 3.5 years, 10 females, mean age: 47.5 ± 14.4 years), with no family history of any neurological disorders or a contraindication to MRI scanning were recruited over a period of two months. Each participant was scanned twice with an MRI 3 T scanner using standard DTI sequences. Spinal Cord Toolbox (SCT) software was used for image post-processing. Data were first corrected for motion artefact, then segmented, registered to a template, and then the DTI metrics were computed. The within-participants coefficients of variation (CV%), the single and average within-participants intraclass correlation coefficients (ICC) and Bland-Altman plots for WM, VC, DC and LC fractional anisotropy (FA), mean diffusivity (MD), axial diffusivity (AD), and radial diffusivity (RD) were determined for the cervical spinal cord (between the 2nd and 5th cervical vertebrae). ResultsDTI metrics showed poor to excellent within-participants reliability for both single and average ICC and moderate to high reproducibility for CV%, all variation dependent on the location of the ROI. The BA plots showed good within-participants agreement between the scan-rescan values. ConclusionResults from this reliability study demonstrate that clinical trials using the DTI technique are feasible and that DTI, in particular regions of the cord is suitable for use for the monitoring of degenerative WM changes.

Construct validity, responsiveness and reliability of the Feline Grimace Scale in kittens.

The aim of the present study was to investigate the construct validity, responsiveness and reliability of the Feline Grimace Scale (FGS) in kittens. A total of 36 healthy female kittens (aged 10 weeks to 6 months) were included in a prospective, randomized, blinded study. Video recordings of all kittens were made before and 1 and 2 h after ovariohysterectomy using an opioid-free injectable anesthetic protocol with or without multimodal analgesia. Additional recordings were taken before and 1 h after administration of rescue analgesia (buprenorphine 0.02 mg/kg IM) to painful kittens. Screenshots of facial images were collected from the video recordings for FGS scoring. Four observers blinded to treatment groups and time points scored 111 randomized images twice with a 5-week interval using the FGS. Five action units (AUs) were scored (ear position, orbital tightening, muzzle tension, whiskers position and head position; 0-2 each). Construct validity, responsiveness, and inter- and intra-rater reliability were evaluated using linear models with Benjamini-Hochberg correction, Wilcoxon signed-rank test and single intraclass correlation coefficients (ICCsingle), respectively (P <0.05). FGS total ratio scores were higher at 1 and 2 h after ovariohysterectomy (median [interquartile range, IQR]: 0.30 [0.20-0.40] and 0.30 [0.20-0.40], respectively) than at baseline (median [IQR]: 0.10 [0.00-0.30]) (P <0.001). FGS total ratio scores were lower after the administration of rescue analgesia (median [IQR] before and after rescue analgesia) 0.40 [0.20-0.50] and 0.20 [0.10-0.38], respectively (P <0.001). Inter-rater ICCsingle was 0.68 for the FGS total ratio scores and 0.35-0.70 for all AUs considered individually. Intra-rater ICCsingle was 0.77-0.91 for the FGS total ratio scores and 0.55-1.00 for all AUs considered individually. The FGS is a valid and responsive acute pain-scoring instrument with moderate inter-rater reliability and good to excellent intra-rater reliability in kittens.

Reliability of a three-dimensional spinal proprioception assessment for patients with adolescent idiopathic scoliosis

PurposeAlthough it is evident that some patients with adolescent idiopathic scoliosis (AIS) have proprioceptive deficit in peripheral joints, knowledge on the proprioceptive function of the deformed spine is limited. Nonetheless, spinal proprioception in AIS may be affected three-dimensionally, prior studies only focussed on evaluating peripheral proprioception in single plane. Therefore, this study aimed to develop a novel spinal proprioception assessment using three-dimensional motion analysis in patients with AIS.MethodsParticipants were included if they had a primary diagnosis of AIS who did not receive or failed conservative treatments. Three trunk repositioning tests involving flexion-extension, lateral-flexion, and axial-rotation were conducted. A three-dimensional kinematics of the trunk was used as the outcome measures. The proprioceptive acuity was quantified by the repositioning error. The intra-examiner and test-retest reliability were analysed by the intraclass correlation coefficient (ICC).ResultsFifty-nine patients with AIS were recruited. Regarding the trunk flexion–extension test, the single measure ICC showed moderate reliability (0.46) and the average measures ICC demonstrated good reliability (0.72). As for the trunk lateral-flexion test, the reliability of single measure and average measures ICC was moderate (0.44) and good (0.70) reliability, respectively. For the trunk axial-rotation test, the single measure ICC indicated fair reliability (0.32), while the average measures ICC showed moderate reliability (0.59).ConclusionThis is the first study to evaluate the reliability of novel three-dimensional spinal proprioception assessments in patients with AIS. The trunk flexion-extension repositioning test may be preferable clinical test given its highest reliability.

Reliability of the freehand region-of-interest method in quantitative cerebral diffusion tensor imaging

BackgroundDiffusion tensor imaging (DTI) is a magnetic resonance imaging (MRI) technique used for evaluating changes in the white matter in brain parenchyma. The reliability of quantitative DTI analysis is influenced by several factors, such as the imaging protocol, pre-processing and post-processing methods, and selected diffusion parameters. The region-of-interest (ROI) method is most widely used of the post-processing methods because it is found in commercial software. The focus of our research was to study the reliability of the freehand ROI method using various intra- and inter-observer analyses.MethodsThis study included 40 neurologically healthy participants who underwent diffusion MRI of the brain with a 3 T scanner. The measurements were performed at nine different anatomical locations using a freehand ROI method. The data extracted from the ROIs included the regional mean values, intra- and inter-observer variability and reliability. The used DTI parameters were fractional anisotropy (FA), the apparent diffusion coefficient (ADC), and axial (AD) and radial (RD) diffusivity.ResultsThe average intra-class correlation coefficient (ICC) of the intra-observer was found to be 0.9 (excellent). The single ICC results were excellent (> 0.8) or adequate (> 0.69) in eight out of the nine regions in terms of FA and ADC. The most reliable results were found in the frontobasal regions. Significant differences between age groups were also found in the frontobasal regions. Specifically, the FA and AD values were significantly higher and the RD values lower in the youngest age group (18–30 years) compared to the other age groups.ConclusionsThe quantitative freehand ROI method can be considered highly reliable for the average ICC and mostly adequate for the single ICC. The freehand method is suitable for research work with a well-experienced observer. Measurements should be performed at least twice in the same region to ensure that the results are sufficiently reliable. In our study, reliability was slightly undermined by artifacts in some regions such as the cerebral peduncle and centrum semiovale. From a clinical point of view, the results are most reliable in adults under the age of 30, when age-related changes in brain white matter have not yet occurred.

Reliability of forearm medial-anterior surface dimensional changes at different isometric hand grip forces

Indirect techniques of predicting hand grip force are fundamental to develop hand control systems for assistive devices. The purpose of this study was to determine the reliability of 3D position of forearm surface at different isometric hand grip forces. Three-dimensional motion analysis was used to measure displacement of 24 discrete and standardized surface markers placed on forearm in 20 healthy participants. The relative displacements were measured for isometric grip forces at 0%, 5%, 20% 50% and 80% of maximum voluntary contraction (MVC). Intraclass correlation coefficient (ICC) for relative radial displacement (RRD) of each marker was calculated. Averaged single measure ICC of 24 markers at five grip forces was 0.61; while the highest averaged single measure ICC (0.80) for all markers was achieved at 80% of MVC and the lowest (0.47) at 0% of MVC. The average measure ICC for each grip force across the 24 markers also increased with grip force from 0.80 at 0% of MVC to the maximum of 0.95 at 80% of MVC. In conclusion, RRD showed moderate and high ICCs for single and average measures respectively. Overall, this study suggests that the reliable dimensional changes of 3D positions of forearm surface might be considered as an indirect and non-invasive method to predict hand grip force in future.

Validation of the Modified Self-Administered Pediatric Bleeding Assessment Questionnaire (Self-PBQ) in Children Aged 8-12 Years

IntroductionIn the pediatric population, pathologic bleeding is often more challenging to identify than in adult patients. Hemorrhagic symptoms are seen in healthy children as well as those with bleeding disorders and children have fewer exposures to events such as surgery that would cause symptoms to manifest. A pediatric bleeding assessment tool may facilitate the accurate evaluation of bleeding symptoms in children.The Pediatric Bleeding Questionnaire (PBQ) was developed as a screening tool for von Willebrand Disease (VWD), the most common hereditary bleeding disorder. The PBQ provides a quantitative score for bleeding symptoms based on the type and severity of episodes. When administered by experts, the PBQ had a sensitivity of 83% and a specificity of 79%. Recently, the expert-administered PBQ was modified to generate a self-administered PBQ which can be completed by children/parents themselves. A prospective multicentre study was undertaken to validate the Self-PBQ and found it to have a high level of agreement with the expert-administered PBQ, with an intraclass correlation (ICC) of 0.917. However, only children above age 12 yrs completed the Self-PBQ by themselves while those under 12 had a parent fill out the Self-PBQ on their behalf.The present study is sub-study of the recent multicentre study. Our objectives were to determine whether a modified school age Self-PBQ can be completed by younger children with minimal adult assistance, and to validate the school age Self-PBQ in the 8-12 year-old population.MethodologyThis sub-study was conducted in two phases. Phase 1 involved pre-testing and optimizing the Self-PBQ to be suitable for 8-12 year-olds. Seven subjects aged 8-11 yrs, 5 with a confirmed diagnosis of type 1 VWD and 2 healthy controls, underwent cognitive debriefing to identify any problems in the questionnaire. Based on respondent feedback, item reduction was performed to simplify the PBQ, and modifications were made to the question wording and response options.In Phase 2, the school age Self-PBQ was tested at 5 participating treatment centres across Canada. Patients aged 8-12 yrs with a known diagnosis of an inherited bleeding disorder were eligible for inclusion. Children completed the Self-PBQ independently or with assistance from their parent and the amount of help required was noted. Parents also filled out the Self-PBQ separately to serve as a comparison. Bleeding scores (BSs) derived from the child self-report were compared to those of the parent proxy. A single measure ICC based on absolute agreement was used to determine the degree of agreement between the child and parent reports. In addition, subgroup analyses were performed to examine whether the correlation differed in younger (8-9 yrs) versus older (10-12 yrs) age groups or based on the severity of the disease. An ICC ≥ 0.70 was considered the minimum standard for reliability based on existing quality criteria.ResultsThirty-one patients aged 8-12 yrs and their parents completed the Self-PBQ. Two pairs were excluded from the analysis because they did not answer portions of the PBQ and their BSs could not be calculated. Total BSs were used for reliability analysis, performed on a final sample size of 29 pairs. Child and parent scores demonstrated a high level of agreement with an ICC of 0.825. In the age subgroup analysis, the ICC was 0.834 and 0.824 for the 8-9 year-old and 10-12 year-old groups respectively. The ICC was also determined in children with type 1 VWD (ICC = 0.829) versus those with more severe bleeding disorders (ICC = 0.802). Thus, age and disease severity had no significant effect on degree of agreement (Figure).ConclusionOur study evaluated the effectiveness of a school age Self-PBQ when completed by children aged 8-12 yrs. The BSs derived from the child self-report were found to have sufficient concordance with those from the parent proxy report. Moreover, the agreement was maintained in younger children aged 8-9 yrs and in children on the more severe spectrum of bleeding disorders. These results indicate that the Self-PBQ is suitable to be incorporated into routine clinical care for the 8-12 year-old population.Figure: Agreement between child and parent (proxy) total reported bleeding scores (BSs). The difference between child and parent-derived BSs is plotted against the mean of the two measurements. The average of the differences is shown by the red line. The numbers plotted reflect ages of the children. [Display omitted] DisclosuresJames:Shire: Research Funding; Bayer: Research Funding; CSL Behring: Research Funding. Klaassen:Baxalta: Honoraria; Agios Pharmaceuticals: Consultancy; Octapharma: Honoraria; Biogen Canada LTD: Consultancy; Amgen: Consultancy; Hoffman-La Roche LTD: Consultancy.

Imputing intracluster correlation coefficients from a posterior predictive distribution is a feasible method of dealing with unit of analysis errors in a meta-analysis of cluster RCTs

BackgroundIncorporating cluster randomized trials (CRTs) into meta-analyses is challenging because appropriate standard errors of study estimates accounting for clustering are not always reported. Systematic reviews of CRTs often use a single constant external estimate of the intraclass correlation coefficient (ICC) to adjust study estimate standard errors and facilitate meta-analyses; an approach that fails to account for possible variation of ICCs among studies and the imprecision with which they are estimated. Using a large systematic review of the effects of diabetes quality improvement interventions, we investigated whether we could better account for ICC variation and uncertainty in meta-analyzed effect estimates by imputing missing ICCs from a posterior predictive distribution constructed from a database of relevant ICCs. MethodsWe constructed a dataset of ICC estimates from applicable studies. For outcomes with two or more available ICC estimates, we constructed posterior predictive ICC distributions in a Bayesian framework. For a selected continuous outcome, glycosylated hemoglobin (HbA1c), we compared the impact of incorporating a single constant ICC versus imputing ICCs drawn from the posterior predictive distribution when estimating the effect of intervention components on post treatment mean in a case study of diabetes quality improvement trials. ResultsUsing internal and external ICC estimates, we were able to construct a database of 59 ICCs for 12 of the 13 review outcomes (range 1–10 per outcome) and estimate the posterior predictive ICC distribution for 11 review outcomes. Synthesized results were not markedly changed by our approach for HbA1c. ConclusionBuilding posterior predictive distributions to impute missing ICCs is a feasible approach to facilitate principled meta-analyses of cluster randomized trials using prior data. Further work is needed to establish whether the application of these methods leads to improved review inferences for different reviews based on different factors (e.g., proportion of CRTs and CRTs with missing ICCs, different outcomes, variation and precision of ICCs).

Detection of early cartilage degeneration in the tibiotalar joint using 3 T gagCEST imaging: a feasibility study

ObjectiveTo establish and optimize a stable 3 Tesla (T) glycosaminoglycan chemical exchange saturation transfer (gagCEST) imaging protocol for assessing the articular cartilage of the tibiotalar joint in healthy volunteers and patients after a sustained injury to the ankle.MethodsUsing Bloch–McConnell simulations, we optimized the sequence protocol for a 3 T MRI scanner for maximum gagCEST effect size within a clinically feasible time frame of less than 07:30 min. This protocol was then used to analyze the gagCEST effect of the articular cartilage of the tibiotalar joint of 17 healthy volunteers and five patients with osteochondral lesions of the talus following ankle trauma. Reproducibility was tested with the intraclass correlation coefficient.ResultsThe mean magnetization transfer ratio asymmetry (MTRasym), i.e., the gagCEST effect size, was significantly lower in patients than in healthy volunteers (0.34 ± 1.9% vs. 1.49 ± 0.11%; p < 0.001 [linear mixed model]). Intra- and inter-rater reproducibility was excellent with an average measure intraclass correlation coefficient (ICC) of 0.97 and a single measure ICC of 0.91 (p < 0.01).DiscussionIn this feasibility study, pre-morphological tibiotalar joint cartilage damage was quantitatively assessable on the basis of the optimized 3 T gagCEST imaging protocol that allowed stable quantification gagCEST effect sizes across a wide range of health and disease in clinically feasible acquisition times.

Toward a Dimensional Assessment of Externalizing Disorders in Children: Reliability and Validity of a Semi-Structured Parent Interview.

ObjectiveThis study assesses the reliability and validity of the DSM-5-based, semi-structured Clinical Parent Interview for Externalizing Disorders in Children and Adolescents (ILF-EXTERNAL).MethodParticipant data were drawn from the ongoing ESCAschool intervention study. The ILF-EXTERNAL was evaluated in a clinical sample of 474 children and adolescents (aged 6−12 years, 92 females) with symptoms of attention-deficit/hyperactivity disorder (ADHD). To obtain interrater reliability, the one-way random-effects, absolute agreement models of the intraclass correlation (ICC) for single ICC(1,1) and average measurements ICC(1,3) were computed between the interviewers and two independent raters for 45 randomly selected interviews involving ten interviewers. Overall agreement on DSM-5 diagnoses was assessed using Fleiss’ kappa. Further analyses evaluated internal consistencies, item-total correlations as well as correlations between symptom severity and the degree of functional impairment. Additionally, parents completed the German version of the Child Behavior Checklist (CBCL) and two DSM-5-based parent questionnaires for the assessment of ADHD symptoms and symptoms of disruptive behavior disorders (FBB-ADHS; FBB-SSV), which were used to evaluate convergent and divergent validity.ResultsICC coefficients demonstrated very good to excellent interrater reliability on the item and scale level of the ILF-EXTERNAL [scale level: ICC(1,1) = 0.83−0.95; ICC(1,3) = 0.94−0.98]. Overall kappa agreement on DSM-5 diagnoses was substantial to almost perfect for most disorders (0.38 ≤ κ ≤ 0.94). With some exceptions, internal consistencies (0.60 ≤ α ≤ 0.86) and item-total correlations (0.21 ≤ rit ≤ 0.71) were generally satisfactory to good. Furthermore, higher symptom severity was associated with a higher degree of functional impairment. The evaluation of convergent validity revealed positive results regarding clinical judgment and parent ratings (FBB-ADHS; FBB-SSV). Correlations between the ILF-EXTERNAL scales and the CBCL Externalizing Problems were moderate to high. Finally, the ILF-EXTERNAL scales were significantly more strongly associated with the CBCL Externalizing Problems than with the Internalizing Problems, indicating divergent validity.ConclusionIn clinically referred, school-age children, the ILF-EXTERNAL demonstrates sound psychometric properties. The ILF-EXTERNAL is a promising clinical interview and contributes to high-quality diagnostics of externalizing disorders in children and adolescents.

Development of the Diabetic Wound Assessment Learning Tool (DiWALT) and validity evidence

ObjectiveDiabetic foot wounds account for up to one-third of diabetes-related health care expenditure and are the greatest cause of extremity amputation in Canada. Physicians encounter patients with such wounds in all specialties, particularly as generalists in medical wards and emergency departments. However, there is a dearth of literature on the optimal way to teach and to assess the management of these patients. Given the importance of assessment for learning in the shift toward competency-based medical education, we aimed to develop an assessment tool and to build validity evidence for its use in this context. MethodsA consensus process involving nine Canadian experts in diabetic wound management was used to develop the Diabetic Wound Assessment Learning Tool (DiWALT) items and two 10-minute simulation-based testing scenarios. The simulators used were modified from commercially available models to serve the testing scenarios. Validity evidence for the DiWALT was subsequently evaluated by assessing 24 physician participants' performance during the two scenarios. All participants were novices (<50 cases managed). Two assessors independently rated participants using the DiWALT. Evidence was organized using Kane's validity framework and included Cronbach α for interitem consistency as well as test-retest and inter-rater reliability using the intra-class correlation coefficient (ICC). ResultsCronbach α was 0.92, implying high internal consistency. Test-retest reliability was also excellent with ICC of 0.89 (confidence interval [CI], 0.76-0.95) for single measures and ICC of 0.94 (CI, 0.86-0.98) for average measures. Inter-rater reliability was fair for single measures with ICC of 0.68 (CI, 0.65-0.71) and good for average measures with ICC of 0.81 (CI, 0.79-0.83). ConclusionsThese results demonstrate that the DiWALT consistently and reliably evaluates competence in diabetic wound management during simulated cases using a small, homogeneous sample of physicians. Further work is necessary to quantify sources of error in the assessment scores, to establish validity evidence when it is used to assess larger and more heterogeneous participants, and to identify how well the DiWALT differentiates between different experience levels.

Automated CT registration tool improves sensitivity to change in ventricular volume in patients with shunts and drains.

CT is the mainstay imaging modality for assessing change in ventricular volume in patients with ventricular shunts or external ventricular drains (EVDs). We evaluated the performance of a novel fully automated CT registration and subtraction method to improve reader accuracy and confidence compared with standard CT. In a retrospective evaluation of 49 ventricular shunt or EVD patients who underwent sequential head CT scans with an automated CT registration tool (CT CoPilot), three readers were assessed on their ability to discern change in ventricular volume between scans using standard axial CT images versus reformats and subtraction images generated by the registration tool. The inter-rater reliability among the readers was calculated using an intraclass correlation coefficient (ICC). Bland-Altman tests were performed to determine reader performance compared to semi-quantitative assessment using the bifrontal horn and third ventricular width. McNemar's test was used to determine whether the use of the registration tool increased the reader's level of confidence. Inter-rater reliability was higher when using the output of the registration tool (single measure ICC of 0.909 with versus 0.755 without the tool). Agreement between the readers' assessment of ventricular volume change and the semi-quantitative assessment improved with the registration tool (limits of agreement 4.1 vs 4.3). Furthermore, the tool improved reader confidence in determining increased or decreased ventricular volume (p < 0.001). Automated CT registration and subtraction improves the reader's ability to detect change in ventricular volume between sequential scans in patients with ventricular shunts or EVDs. Our automated CT registration and subtraction method may serve as a promising generalizable tool for accurate assessment of change in ventricular volume, which can significantly affect clinical management.

Intra-individual variations of organophosphate pesticide metabolite concentrations in repeatedly collected urine samples from pregnant women in Japan

BackgroundLow-dose exposure to organophosphate (OP) insecticides during pregnancy may adversely affect neurodevelopment in children. To evaluate the OP exposure levels, single urine sampling is commonly adopted to measure the levels of dialkylphosphates (DAPs), common OP metabolites. However, the inter-day variations of urinary DAP concentrations within subjects are supposed to be large due to the short biological half-lives of the metabolites, and it is thus considered difficult to accurately assess OP exposure during pregnancy with single sampling. This study aimed to assess intra-individual variations of DAP concentrations and the reproducibility of the exposure dose categorization of OPs according to DAP concentration ranges in pregnant women in Japan.MethodsUrine samples were collected from 62 non-smoking pregnant women (12–22 weeks of gestation) living in Aichi Prefecture, Japan. First morning void (FMV) and spot urine samples taken between lunch and dinner on the same day were collected on five different days during 2 weeks. The concentrations of DAP and creatinine in urine samples were measured using an ultra performance liquid chromatography with tandem mass spectrometry. Creatinine-adjusted and unadjusted concentrations were used for the intraclass correlation coefficient (ICC) calculations and surrogate category analyses.ResultsFor all DAP metabolites, the creatinine-adjusted single ICCs exceeded 0.4, indicating moderate reliability. Overall, ICCs of spot urine samples taken in the afternoon were better than those taken as FMV. Surrogate category analyses showed that participants were categorized accurately into four exposure dose groups according to the quartile points.ConclusionThis study indicated that a single urine sample taken in the afternoon may be useful in assessing OP exposure as long as the exposure is categorized into quartiles when conducting epidemiological studies in early to mid-pregnant women in Japan.

Reliability of Using Motion Sensors to Measure Children's Physical Activity Levels in Exergaming.

Objectives: This study examined the reliability of two objective measurement tools in assessing children’s physical activity (PA) levels in an exergaming setting. Methods: A total of 377 children (190 girls, Mage = 8.39, SD = 1.55) attended the 30-min exergaming class every other day for 18 weeks. Children’s PA levels were concurrently measured by NL-1000 pedometer and ActiGraph GT3X accelerometer, while children’s steps per min and time engaged in sedentary, light, and moderate-to-vigorous PA were estimated, respectively. Results: The results of intraclass correlation coefficient (ICC) indicated a low degree of reliability (single measures ICC = 0.03) in accelerometers. ANOVA did detect a possible learning effect for 27 classes (p < 0.01), and the single measures ICC was 0.20 for pedometers. Moreover, there was no significant positive relationship between steps per min and time spent in moderate-to-vigorous physical activity (MVPA). Finally, only 1.3% variance was explained by pedometer as a predictor using Hierarchical Linear Modeling to further explore the relationship between pedometer and accelerometer data. Conclusions: The NL-1000 pedometers and ActiGraph GT3X accelerometers have low reliability in assessing elementary school children’s PA levels during exergaming. More research is warranted in determining the reliable and accurate measurement information regarding the use of modern devices in exergaming setting.

Manual muscle testing and hand-held dynamometry in people with inflammatory myopathy: An intra- and interrater reliability and validity study.

Manual muscle testing (MMT) and hand-held dynamometry (HHD) are commonly used in people with inflammatory myopathy (IM), but their clinimetric properties have not yet been sufficiently studied. To evaluate the reliability and validity of MMT and HHD, maximum isometric strength was measured in eight muscle groups across three measurement events. To evaluate reliability of HHD, intra-class correlation coefficients (ICC), the standard error of measurements (SEM) and smallest detectable changes (SDC) were calculated. To measure reliability of MMT linear Cohen`s Kappa was computed for single muscle groups and ICC for total score. Additionally, correlations between MMT8 and HHD were evaluated with Spearman Correlation Coefficients. Fifty people with myositis (56±14 years, 76% female) were included in the study. Intra-and interrater reliability of HHD yielded excellent ICCs (0.75–0.97) for all muscle groups, except for interrater reliability of ankle extension (0.61). The corresponding SEMs% ranged from 8 to 28% and the SDCs% from 23 to 65%. MMT8 total score revealed excellent intra-and interrater reliability (ICC>0.9). Intrarater reliability of single muscle groups was substantial for shoulder and hip abduction, elbow and neck flexion, and hip extension (0.64–0.69); moderate for wrist (0.53) and knee extension (0.49) and fair for ankle extension (0.35). Interrater reliability was moderate for neck flexion (0.54) and hip abduction (0.44); fair for shoulder abduction, elbow flexion, wrist and ankle extension (0.20–0.33); and slight for knee extension (0.08). Correlations between the two tests were low for wrist, knee, ankle, and hip extension; moderate for elbow flexion, neck flexion and hip abduction; and good for shoulder abduction. In conclusion, the MMT8 total score is a reliable assessment to consider general muscle weakness in people with myositis but not for single muscle groups. In contrast, our results confirm that HHD can be recommended to evaluate strength of single muscle groups.

Problems With Reliability of Speech Variables for Use in Quality Registries for Cleft Lip and Palate-Experiences From the Swedish Cleft Lip and Palate Registry.

To test the reliability of the speech data in the Swedish quality registry for cleft lip and palate. Retrospective study. Primary care university hospital. Ninety-four children born with cleft palate with or without cleft lip between 2005 and 2009 who had been assessed and registered in the quality registry at the age of 5 years. Data in the registry on percent oral consonants correct, percent oral errors, percent nonoral errors, perceived velopharyngeal function (PVPF), and intelligibility were compared with results based on reassessments by 3 independent raters from audio recordings. Agreement was calculated by the intraclass correlation coefficient (ICC), quadratic weighted kappa, and percentage agreement. Absolute agreement calculated by average measures ICC for percent oral consonants correct, percent oral errors, and percent nonoral errors was above >0.90. Single measures ICC for percent oral consonants correct was 0.82, for percent oral errors 0.69, and for percent nonoral errors 0.83. The kappa coefficient for PVPF was 0.5 to 0.59 and for intelligibility 0.65 to 0.77. Exact percentage agreement for PVPF was 33% and for intelligibility 47.8%. The data on oral consonants correct and nonoral errors in the quality registry seem to be reliable. The data on oral errors, PVPF, and intelligibility should be interpreted with caution. If differences among treatment centers are detected, one should go back and examine the collected raw data before drawing any definitive conclusions about treatment outcome.

Psychometric properties of the German version of Observer OPTION5

BackgroundIn order to conduct studies on shared decision-making (SDM) and to implement SDM in routine practice, psychometrically tested measures are needed. The development of the short 5-item version of the OPTION scale (Observer OPTION5) allows to assess SDM from an observer perspective. Observer OPTION5 is so far only available in English and Dutch. The aim of this study was to translate the Observer OPTION5 rating scale into German and to test its psychometric properties.MethodsThe German Observer OPTION5 was tested in a secondary data analysis of audio-recordings of patient-physician-consultations (N = 79) in German primary care practices. Demographic data were analysed using descriptive statistics. To assess inter- and intra-rater reliability, intraclass correlation coefficients (ICCs) were calculated. For assessing concurrent validity, a correlation (Spearman’s Rho) of the sum score of Observer OPTION5 and Observer OPTION12 was calculated.ResultsThe consultations dealt with decisions regarding type 2 diabetes (N = 31), chronic back pain (N = 23), depression (N = 20), and other diseases (N = 5). Analysis of inter-rater reliability yielded an ICC of 0.82 for the sum score; across the five single items ICCs ranged between 0.45 and 0.77. For the intra-rater reliability an ICC of 0.83 was observed for the total score; across the five single items ICCs ranged between 0.45 and 0.86. The Observer OPTION5 had a mean total score of 11.84 (SD = 11.92) and the Observer OPTION12 had a mean total score of 10.3 (SD = 7.9), both on a potential range of 0 to 100. The correlation between the total scores of Observer OPTION5 and Observer OPTION12 was r = 0.47 (p = 0.01).ConclusionsThe results regarding inter- and intra-rater reliability were excellent on the total score level. Observer OPTION5 showed moderate concurrent validity using Observer OPTON12. The results are generally comparable to the results of the original English version of Observer OPTION5. The German version of Observer OPTION5 can be used in research and evaluation of clinical practice. Nevertheless, further testing is adviced.

Consistency of fNIRS parameters: A parallel forms and test-retest reliability study

Consistency of fNIRS parameters: A parallel forms and test-retest reliability study

Development of an ethogram to describe facial expressions in ridden horses (FEReq)

Many horses presumed to be sound by their riders are not. Facial expression ethograms have previously been used to describe pain-related behavior in horses, but there is a need for a ridden horse facial ethogram to facilitate identification of pain in ridden horses. The objectives of this study were to develop and test an ethogram to describe facial expressions in ridden horses and to determine whether individuals could interpret and correctly apply the ethogram, with consistency among assessors. An ethogram was developed by reference to previous publications and photographs of 150 lame and nonlame ridden horses. A training manual was created. Thirteen assessors (veterinarians of variable experience, n = 4; equine technicians, n = 3; equine studies graduates, n = 2; amateur horse owners, n = 2; equine veterinary nurse, n = 1; a British Horse Society Instructor, n = 1) underwent a training session and, with reference to the training manual, evaluated still lateral photographs of 27 training heads. Features were graded as Yes, No, or “Cannot see” (when it was not possible to determine the presence or absence of a feature). The ethogram was adapted, and after further training, the assessors blindly evaluated 30 test heads from nonlame and lame horses. Intraclass correlation (ICC) and free-margin kappa tests were used to assess consensus among assessors. For the training heads, single ICC matrix among observers resulted in an overall ICC of 0.50 (95% confidence intervals, 0.40-0.62). Four assessors consistently scored differently from the others, with ranges of ICC of 0.20-0.50 (mean, 0.41). There was no difference in assessors' scoring related to their professional backgrounds. For the test heads, mean interrater agreement among assessors was 87%. Two assessors still scored consistently differently (0.28-0.50 ICC agreement; mean, 0.40) from the remaining 11 assessors (0.44-0.69 ICC agreement; mean, 0.56). The mean percentage of overall agreement was 80%, and the mean free-marginal kappa value was 0.72, standard deviation (SD) ± 0.22. The large SD was the result of inconsistency in assessments of the eyes and muzzle. It was concluded that the developed ethogram could reliably be utilized to describe facial expressions of ridden horses by people from different professional backgrounds. Future work needs to determine if nonlame and lame horses can be differentiated based on application of the ethogram.

Development and interrater reliability of the UK Mental Health Triage Scale.

Mental health triage scales are clinical tools used at point of entry to specialist mental health service to provide a systematic way of categorizing the urgency of clinical presentations, and determining an appropriate service response and an optimal timeframe for intervention. The aim of the present study was to test the interrater reliability of a mental health triage scale developed for use in UK mental health triage and crisis services. An interrater reliability study was undertaken. Triage clinicians from England and Wales (n = 66) used the UK Mental Health Triage Scale (UK MHTS) to rate the urgency of 21 validated mental health triage scenarios derived from real occasions of triage. Interrater reliability was calculated using Kendall's coefficient of concordance (w) and intraclass correlation coefficient (ICC) statistics. The average ICC was 0.997 (95% confidence interval (CI): 0.996-0.999 (F (20, 1300) = 394.762, P < 0.001). The single measure ICC was 0.856 (95% CI: 0.776-0.926 (F (20, 1300) = 394.762, P < 0.001). The overall Kendall's w was 0.88 (P < 0.001). The UK MHTS shows substantial levels of interrater reliability. Reliable mental health triage scales employed within effective mental health triage systems offer possibilities for not only improved patient outcomes and experiences, but also for efficient use of finite specialist mental health services.

Accelerometer data requirements for reliable estimation of habitual physical activity and sedentary time of children during the early years - a worked example following a stepped approach

ABSTRACTThis study presents a worked example of a stepped process to reliably estimate the habitual physical activity and sedentary time of a sample of young children. A total of 299 children (2.9 ± 0.6 years) were recruited. Outcome variables were daily minutes of total physical activity, sedentary time, moderate to vigorous physical activity and proportional values of each variable. In total, 282 (94%) provided 3 h of accelerometer data on ≥1 day and were included in a 6-step process: Step-1: determine minimum wear-time; Step-2: process 7-day-data; Step-3: determine the inclusion of a weekend day; Step-4: examine day-to-day variability; Step-5: calculate single day intraclass correlation (ICC) (2,1); Step-6: calculate number of days required to reach reliability. Following the process the results were, Step-1: 6 h was estimated as minimum wear-time of a standard day. Step-2: 98 (32%) children had ≥6 h wear on 7 days. Step-3: no differences were found between weekdays and weekend days (P ≥ 0.05). Step-4: no differences were found between day-to-day variability (P ≥ 0.05). Step-5: single day ICC’s (2,1) ranged from 0.48 (total physical activity and sedentary time) to 0.53 (proportion of moderate to vigorous physical activity). Step-6: to reach reliability (ICC = 0.7), 3 days were required for all outcomes. In conclusion following a 7 day wear protocol, ≥6 h on any 3 days was found to have acceptable reliability. The stepped-process offers researchers a method to derive sample-specific wear-time criterion.