Average Measures Intraclass Correlation Coefficient Research Articles

Intra- and Interobserver Reliability Comparison of Clinical Gait Analysis Data between Two Gait Laboratories

Comparing clinical gait analysis (CGA) data between clinical centers is critical in the treatment and rehabilitation progress. However, CGA protocols and system configurations, as well as choice of marker sets and individual variability during marker attachment, may affect the comparability of data. The aim of this study was to evaluate reliability of CGA data collected between two gait analysis laboratories. Three healthy subjects underwent a standardized CGA protocol at two separate centers. Kinematic data were captured using the same motion capturing systems (two systems, same manufacturer, but different analysis software and camera configurations). The CGA data were analyzed by the same two observers for both centers. Interobserver reliability was calculated using single measure intraclass correlation coefficients (ICC). Intraobserver as well as between-laboratory intraobserver reliability were assessed using an average measure ICC. Interobserver reliability for all joints (ICCtotal = 0.79) was found to be significantly lower (p < 0.001) than intraobserver reliability (ICCtotal = 0.93), but significantly higher (p < 0.001) than between-laboratory intraobserver reliability (ICCtotal = 0.55). Data comparison between both centers revealed significant differences for 39% of investigated parameters. Different hardware and software configurations impact CGA data and influence between-laboratory comparisons. Furthermore, lower intra- and interobserver reliability were found for ankle kinematics in comparison to the hip and knee, particularly for interobserver reliability.

Minimum accelerometer wear-time for reliable estimates of physical activity and sedentary behaviour of people receiving haemodialysis

BackgroundLow levels of physical activity are implicated in low life expectancies of people receiving maintenance haemodialysis. Accelerometers are increasingly being used to quantify activity behaviours of this population but guidance to quality-assure such data is lacking. The objective of this study was to provide data processing and reduction recommendations to ensure accelerometer-derived outcomes are sufficiently reliable for interpretative analysis.MethodsSeventy people receiving maintenance haemodialysis (age 55.9 ± 15.7 years, 34% women, 23% diabetic) from a single outpatient renal unit volunteered for the study. Participants wore Actigraph GT3x and ActivPAL monitors during waking hours over seven days. Reliability of accelerometer output (normalised to wear-time) was assessed via intraclass correlation coefficient (ICC). The Spearman-Brown prophecy formula was subsequently applied to the ICCs to derive the minimum required accelerometer wear-time for each behavioural outcome.ResultsMonitor wear compliance was greater on dialysis compared to non-dialysis days (90% v 77%). Participants were significantly more active on non-dialysis days compared to dialysis days but there were no significant differences in estimated behaviours between days within the same condition. Average measure ICCs for all accelerometer outcomes were high (range 0.76–0.96). Computations indicated that habitual physical activity and sedentary behaviour could be estimated with a minimum reliability level of 0.80 from one dialysis day and two non-dialysis days, and at least eight hours monitor wear per day. Applying this rubric allowed 90% of participant data to be retained for further analysis.ConclusionsRegardless of accelerometer, one dialysis and two non-dialysis days data with a minimum of eight hours wear each day should enable habitual activity of people receiving maintenance haemodialysis to be characterised with acceptable reliability. These recommendations reconcile the tension between wear-time criteria stringency and retention of an adequately representative sample.

Validation of the Safer Opioid Prescribing Evaluation Tool (SOPET) for Assessing Adherence to the Centers for Disease Control Opioid Prescribing Guidelines.

In response to the opioid epidemic, the Centers for Disease Control and Prevention issued guidelines (CDCG) in 2016 for the prescription of opioids for chronic pain. To facilitate research into whether CDCG implementation will lead to reductions in opioid prescribing and improved patient safety, we sought to validate a tool that quantifies CDCG adherence based on clinical documentation. The Safe Opioid Prescribing Evaluation Tool (SOPET) was developed in four phases as part of a study to improve the implementation of the CDCG in the clinical setting. Four raters with varying levels of clinical experience and expertise were trained to use the SOPET and then used it to evaluate 21 baseline patient encounters. Intraclass correlation coefficient (ICC) estimates and their 95% confident intervals (CIs) were calculated for the total SOPET score based on a mean-rating (k = 4), absolute-agreement, two-way random-effects model. For intrarater reliability, two-way mixed-effect models were used. Inter-rater reliability was good, with an average-measures ICC of 0.82 (95% CI = 0.63-0.92). Intrarater reliability was excellent for the three raters, who were MDs, with average-measures ICCs as follows: 0.92 (95% CI = 0.81-0.97), 0.97 (95% CI = 0.92-0.99), 0.99 (95% CI = 0.99-0.99). However, the intrarater reliability for the non-MD rater was lower 0.69 (95% CI = 0.22-0.88). Overall, the SOPET is useful for evaluating implementation of the CDCG in clinical documentation. It is an important first step in the design of future studies assessing whether adherence to the CDCG improves patient safety outcomes.

Cross-cultural adaptation, translation, and psychometric properties of Gujarati version of physical activity questionnaire for older children

BACKGROUND: The Physical Activity Questionnaire for Older Children (PAQ-C) is a self-administered questionnaire to assess physical activity (PA) among older children, and it is available in English, Dutch, Malay, Spanish, Turkish, and Gujarati languages. Cross-cultural adaptation and robust research on psychometric properties arise the need for current research on the PAQ-C in Gujarati language (PAQ-C [G]). METHODS: The study was carried out in three phases: cultural adaptation and translation into the Gujarati language from English using the forward-backward-forward method, pilot study on 10 children to assess the comprehensibility of the prefinal version, and psychometric properties of the final version of the questionnaire. To ensure the face and content validity of the PAQ-C (G) group, a consensus method was used. Each item was examined by a group of experts having a mean experience of 22.11 years in different fields. Each item was analyzed for content, meaning, wording, format, ease of administration, and scoring. Each item was scored by an expert group as either rejected, accepted, or accepted with modification. The procedure was continued until 80% of consensus for all items. Finally, 20 children were asked to complete the PAQ-C (G) on two occasions 1 week apart to examine its test–retest reliability. RESULTS: The reliability coefficient using the average-measure intraclass correlation coefficient (ICC) total score of PAQ-C (G) was 0.82 which is a strong positive correlation. The test–retest between individual items shows that PAQ-C (G) had a moderate-to-strong correlation with ICC value between 0.78 and 0.91. The internal consistency reliability of PAQ-C (G) was α = 0.94 and α = 0.92 for assessments 1 and 2, respectively. CONCLUSION: PAQ-C (G) has a good face–content validity and test–retest reliability which can be used to measure PA among Gujarati speaking older schoolchildren.

Quantification and Repeatability of Vessel Density and Flux as Assessed by Optical Coherence Tomography Angiography.

PurposeTo determine the intrasession repeatability (test-retest variability) of parafoveal and peripapillary perfused capillary density (PCD) and normalized flux index (NFI) as assessed with Canon OCT-HS100 angiography.MethodsPairs of optical coherence tomography angiography (OCT-A) images were obtained from the parafoveal and peripapillary region of 30 eyes of 30 healthy subjects. PCD and NFI were calculated using generic image-processing software. Macular ganglion-cell complex thickness (GCC) and peripapillary retinal nerve fiber layer thickness (RNFLT) were also recorded. Bland-Altman analysis was performed and the coefficient of repeatability (CoR) and intraclass correlation coefficient (ICC) were calculated. Correlations of parafoveal PCD/NFI with GCC and of peripapillary PCD/NFI with RNFLT were also computed.ResultsMean (standard deviation) parafoveal and peripapillary PCD were 40.0% (1.8%) and 44.5% (1.3%), respectively. Corresponding values for NFI were 151.2 (6.8) and 164.2 (3.9). For PCD, ICC was 0.76 for parafoveal and 0.79 for peripapillary measurements; corresponding CoRs were 2.7% and 1.8%. Corresponding values for NFI were 0.62 and 0.67 for ICC and 13.3 and 7.0 for CoR. Average measures ICC was 0.87/0.88 and 0.76/0.80 for the parafoveal/peripapillary PCD and NFI, respectively. PCD and NFI were weakly correlated with GCC (r = 0.39, P = 0.035; r = 0.33, P = 0.077) and moderately correlated with RNFLT (r = 0.43, P = 0.017; r = 0.55, P = 0.002).ConclusionsRepeatability of a commercially available OCT-A with generic image-processing software was good (NFI) to excellent (PCD). Our results indicate that changes surpassing the variability in healthy subjects should be easily detectable in a clinical setting.Translational RelevanceRepeatability estimates provide information regarding the relevance of changes in retinal perfusion.

Examining criterion a: DSM-5 level of personality functioning as assessed through life story interviews.

The Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition alternative model for personality disorders offers a two-part definition of personality pathology, separating personality functioning from traits. The Level of Personality Functioning Scale (LPFS) from the Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition alternative model for personality disorders encapsulates the personality functioning criterion, and several methods have been used to assess it. Previous interview rating methods have overlapped with an assessment of personality traits and symptoms, biasing the assessment of functioning, and recently developed self-report instruments rely on the participant's awareness of their personality pathology. The purpose of the current analyses was to examine the reliability and validity of LPFS ratings based on open-ended, nondiagnostic interviews. The sample consisted of 162 community-dwelling, older adult participants from the St. Louis Personality and Aging Network. Undergraduate students rated video recordings of Life Story Interviews, using a 12-item version of the LPFS. One-way random, average measures intraclass correlation coefficient for the total LPFS was .80. A principal components analysis indicated that a single underlying dimension could characterize the LPFS. Component scores derived from this analysis demonstrated theoretically consistent associations with both normal-range and maladaptive personality traits. The component scores also contributed small but significant variance to the prediction of personality disorder symptoms, health, and functional outcomes over and above personality traits. These findings support the reliability and validity of the LPFS as assessed using Life Story Interviews and suggest that personality functioning ratings may have utility in predicting clinically relevant outcomes. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

Dentin hypersensitivity monitored by cold air quantitative sensory testing.

Quantification of dentin hypersensitivity (DH) is challenging and requires standardised, graded stimulation by natural-like stimuli. The present study aimed at identifying DH subjects and longitudinally monitoring their pain thresholds by cold air quantitative sensory testing (QST). Subject recruitment started with an online DH questionnaire. Respondents were screened by dental air stimulation. Sensitising and habituating subjects were excluded. A recently developed stimulation device was employed for cold air QST. Single tooth DH was verified by applying an equi-intense stimulus to a control tooth. Descriptive statistics were applied for subject characteristics. Mean values were calculated for the stimulation parameters temperature and air flow. Reliability of temperatures for detecting pain and for evoking moderate pain over multiple time points within a 3-week period was analysed by two-way random single- and average-measure intra-class correlation coefficients. A total of 353 persons completed the online DH questionnaire of which 117 were screened. Forty-four passed the screening, yet 15 were excluded for various reasons. Twenty-nine subjects were monitored by QST across 3weeks. Results revealed a high intra-individual stability of the temperature inducing moderate to strong pain intensity (MPI) (single-measure ICC of TMPI 0.83, P<0.001). Mean TMPI was -13.69°C, yet it highly varied among the 29 subjects (SD±10.04°C). Using a novel approach, namely dental QST based on cold air stimuli, we present evidence for temporally stable DH perceptions over a 3-week period. The method fulfils international guideline requirements and is recommendable for obtaining valid results when testing various interventions for DH management.

Subcostal View-Based Longitudinal Strain in Patients With Breast Cancer Is an Alternative to Conventional Apical View-Based Longitudinal Strain

Strain imaging is a robust clinical tool in cardiac surveillance of patients with breast cancer. However, image quality and therefore ability to accurately measure strain are often limited in this patient group because of tissue expanders, implants, and/or flap reconstruction. The aim of the present study was to evaluate the feasibility of measuring left ventricular longitudinal strain (LVLS) in the echocardiographic subcostal view in female patients with breast cancer. A total of 110 studies from 68 female patients with breast cancer were included. The feasibility of LVLS speckle-tracking measurements in the apical three-chamber (3C) and four-chamber (4C) views and in the subcostal 3C and 4C views was evaluated. The LVLS speckle-tracking measurements obtained in these two echocardiographic views were compared using intraclass correlation coefficients and Bland-Altman analyses. The feasibility of LVLS in the apical 3C and 4C views was 98.1% and 98.1%, respectively. In comparison, the feasibility of LVLS in the subcostal 3C and 4C views was 93.6% and 96.3%, respectively. A high degree of reliability was found between apical and subcostal LVLS 3C and 4C measurements. For the 3C view, the average measure intraclass correlation coefficient was 0.81 (95% CI, 0.72-0.88). For the 4C view, the average measure intraclass correlation coefficient was 0.80 (95% CI, 0.70-0.87). Bland-Altman analysis showed good agreement between apical and subcostal measurements in both the 3C and 4C views. Subcostal 3C and 4C LVLS can be reliably measured, with good agreement with conventional LVLS from the apical views, in female patients with breast cancer. Importantly, the subcostal view may provide a novel alternative for trending LVLS in patients with breast cancer who have technically limited apical windows.

Longitudinal Reliability of the OMERACT Thumb Base Osteoarthritis Magnetic Resonance Imaging Scoring System (TOMS).

To assess the longitudinal reliability of the Outcome Measures in Rheumatology (OMERACT) Thumb base Osteoarthritis Magnetic resonance imaging (MRI) Scoring system (TOMS). Paired MRI of patients with hand osteoarthritis were scored in 2 exercises (6-mo and 2-yr followup) for synovitis, subchondral bone defects (SBD), osteophytes, cartilage assessment, bone marrow lesions (BML), and subluxation. Interreader reliability of delta scores was assessed. Little change occurred. Average-measure intraclass correlation coefficients were good-excellent (≥ 0.71), except synovitis (0.55-0.83) and carpometacarpal-1 osteophytes/cartilage assessment (0.47/0.39). Percentage exact/close agreement was 52-92%/68-100%, except BML in 2 years (28%/64-76%). Smallest detectable change was below the scoring increment, except in SBD and BML. TOMS longitudinal reliability was moderate-good. Limited change hampered assessment.

PO023 Cardiovascular Magnetic Resonance Imaging Compared To Echocardiography For Detecting Doxorubicin-Induced Cardiotoxicity

Strain imaging is a robust clinical tool in cardiac surveillance of patients with breast cancer. However, image quality and therefore ability to accurately measure strain are often limited in this patient group because of tissue expanders, implants, and/or flap reconstruction. The aim of the present study was to evaluate the feasibility of measuring left ventricular longitudinal strain (LVLS) in the echocardiographic subcostal view in female patients with breast cancer.A total of 110 studies from 68 female patients with breast cancer were included. The feasibility of LVLS speckle-tracking measurements in the apical three-chamber (3C) and four-chamber (4C) views and in the subcostal 3C and 4C views was evaluated. The LVLS speckle-tracking measurements obtained in these two echocardiographic views were compared using intraclass correlation coefficients and Bland-Altman analyses.The feasibility of LVLS in the apical 3C and 4C views was 98.1% and 98.1%, respectively. In comparison, the feasibility of LVLS in the subcostal 3C and 4C views was 93.6% and 96.3%, respectively. A high degree of reliability was found between apical and subcostal LVLS 3C and 4C measurements. For the 3C view, the average measure intraclass correlation coefficient was 0.81 (95% CI, 0.72–0.88). For the 4C view, the average measure intraclass correlation coefficient was 0.80 (95% CI, 0.70–0.87). Bland-Altman analysis showed good agreement between apical and subcostal measurements in both the 3C and 4C views.Subcostal 3C and 4C LVLS can be reliably measured, with good agreement with conventional LVLS from the apical views, in female patients with breast cancer. Importantly, the subcostal view may provide a novel alternative for trending LVLS in patients with breast cancer who have technically limited apical windows.

Pediatric-Oncology Simulation Training for Resident Education.

We sought to evaluate pediatric oncology simulations intended to improve pediatric residents' skills and comfort in caring for children with cancer. In a non-randomized trial, controls (the first three rotations) received a standard set of lectures, and the intervention arm received these lectures plus five simulation-training scenarios-fever/neutropenia, a new leukemia diagnosis, end-of-life care discussion, tumor lysis syndrome, and a mediastinal mass. All residents were tested after the rotation on the first three scenarios; management skills were evaluated independently by two raters. Before and after training, all residents completed an emotional-appraisal questionnaire evaluating each scenario as a perceived challenge or threat. Analysis of variance (ANOVA) measured differences by study arm in skills-checklist assessments and appraisals; repeated-measures ANOVA measured changes in emotional-appraisal scores. Forty-two residents (9 control, 33 intervention) participated. Inter-rater agreement for skills-checklist scores using average-measures intraclass correlation was high (0.847), and overall mean scores were significantly higher for the intervention than control group across both raters (P = 0.005). For all residents, perceived challenge increased in the end-of-life simulation, and perceived threat decreased in all three test scenarios. The intervention group, regardless of training year, evaluated the teaching scenarios favorably and felt that challenging oncology situations were addressed, skills were enhanced, and the simulations should be offered to other residents. It was feasible to introduce residents to difficult pediatric oncology scenarios using simulation. The intervention group performed more skills than controls when tested, and perceive threat declined in all residents after their pediatric oncology rotation.

Item distribution and inter-rater reliability of the German version of the quality of life in Alzheimer\u2019s disease scale (QoL-AD) proxy for people with dementia living in nursing homes

BackgroundThe Quality of Life in Alzheimer’s disease scale (QoL-AD) is a widely used Health Related Quality of Life (HRQoL) instrument. However, studies investigating the instrument’s inter-rater reliability (IRR) are missing. This study aimed to determine the item distribution and IRR of the German proxy version of the QoL-AD (13 Items) and a nursing home-specific instrument version (QoL-AD NH, 15 Items).MethodsThe instruments were applied to 73 people with dementia living in eight nursing homes in Germany. Individuals with dementia were assessed two times by blinded proxy raters. The IRR analyses were based on methodological criteria of the quality appraisal tool for studies of diagnostic reliability (QAREL), the COSMIN group and the single-measure Intra-Class Correlation Coefficient (ICC) for absolute agreement ≥0.70.ResultsAll items for both instrument versions demonstrated acceptable item difficulty, with the exception of one item (QoL-AD proxy). The IRR was moderate for the QoL-AD (ICC: 0.65) and insufficient for the QoL-AD NH (ICC: 0.18). The additional computation of the average measure ICC for two proxy-raters demonstrated a strong IRR (ICC: 0.79) for the QoL-AD and a weak IRR for the QoL-AD NH (ICC: 0.31). The detailed analysis of the IRR for each item underpinned the need for the further development of both instruments.ConclusionsThe unsatisfactory IRRs for both instruments highlight the need for the development of a user guide including general instructions for instrument application as well as definitions and examples reflecting item meaning. Priority should be given to the development of reliable proxy-person versions of both instruments.Trial registrationClinicalTrials.gov: NCT02295462, Date of registration: 11–20-2014.

Understanding Motivations for Gardening Using a Qualitative General Inductive Approach

Although some benefits of gardening have been documented, motivations regarding participation in gardening are often considered based on anecdote. The purpose of this study was to use qualitative analysis to explore reasons gardeners from different genders and generations participate in gardening. The questions developed for this study were intentionally exploratory and left open-ended to gather a large variety of responses. Surveys were collected from 177 individuals between the ages of 7 and 94 years old. Responses were categorized into themes identified through the literature review, the pilot study, and through exploration of the data. Responses could fit into as many categories as were mentioned by the respondents and were categorized by three independent coders. Interrater reliability was assessed using a two-way mixed, absolute agreement, average measures intraclass correlation (ICC) and determined the degree to which coders provided consistency in their ratings across participants. Themes developed through this survey included “social interaction,” “aesthetics,” “food availability/health/nutrition,” “economics,” “therapeutic,” “environmental benefits,” “nostalgia,” and “personal productivity.” Themes of personal productivity and nostalgia are those which have not occurred in previous research. Statistically significant differences were found in comparisons among males and females with more males gardening for food/health/nutrition and for reasons regarding nostalgia. More females reported gardening for personal productivity when compared with males. No significant difference was identified in comparisons of gardeners from various age groups indicating that gardeners across generations have similar intentions and receive similar benefits.

Problems With Reliability of Speech Variables for Use in Quality Registries for Cleft Lip and Palate-Experiences From the Swedish Cleft Lip and Palate Registry.

To test the reliability of the speech data in the Swedish quality registry for cleft lip and palate. Retrospective study. Primary care university hospital. Ninety-four children born with cleft palate with or without cleft lip between 2005 and 2009 who had been assessed and registered in the quality registry at the age of 5 years. Data in the registry on percent oral consonants correct, percent oral errors, percent nonoral errors, perceived velopharyngeal function (PVPF), and intelligibility were compared with results based on reassessments by 3 independent raters from audio recordings. Agreement was calculated by the intraclass correlation coefficient (ICC), quadratic weighted kappa, and percentage agreement. Absolute agreement calculated by average measures ICC for percent oral consonants correct, percent oral errors, and percent nonoral errors was above >0.90. Single measures ICC for percent oral consonants correct was 0.82, for percent oral errors 0.69, and for percent nonoral errors 0.83. The kappa coefficient for PVPF was 0.5 to 0.59 and for intelligibility 0.65 to 0.77. Exact percentage agreement for PVPF was 33% and for intelligibility 47.8%. The data on oral consonants correct and nonoral errors in the quality registry seem to be reliable. The data on oral errors, PVPF, and intelligibility should be interpreted with caution. If differences among treatment centers are detected, one should go back and examine the collected raw data before drawing any definitive conclusions about treatment outcome.

In vitro evaluation of color and translucency reproduction of maxillofacial prostheses using a computerized system.

PURPOSEAccurate color matching of maxillofacial prostheses to skin is important for esthetics. A computerized color matching system specific to human skin has recently been developed. The purpose of this study was to evaluate the accuracy in color and translucency matching of the computerized color matching system across different skin colors.MATERIALS AND METHODSThe silicone was colored to simulate 28 different skin colors (n=5) to serve as “target skin colors”. Using a spectrocolorometer (e-skin), color codes were determined for “replicate skin color” fabrication. CIELAB Delta-E between target skin color-replicate skin color pairs and translucency parameter were calculated. CIELAB Delta-E values were compared with one-way ANOVA and Tukey multiple-comparison. The agreement between L*, a*, b* and translucency parameter of target skin colors and replicate skin color were calculated by a two-way mixed average measures intraclass correlation coefficient. Translucency parameter of target skin color-replicate skin color pairs were compared with Paired t-test (α=.05).RESULTSThe mean CIELAB Delta-E value was 3.83 and significant differences were found among colors. The intraclass correlation coefficient showed excellent reliability for L*, a*, b* and good reliability for translucency parameter (P<.001). The mean translucency parameter of replicate skin colors was significantly higher than that of translucency parameter.CONCLUSIONThe computerized color matching system specific to human skin was found to be reliable in terms of color and translucency between target skin colors and replicate skin color.

INTRA-SESSION RELIABILITY AND REPEATABILITY OF KNEE KINEMATICS IN SUBJECTS WITH ACL DEFICIENCY DURING STAIR ASCENT

Analysis of knee kinematics and ground reaction forces (GRFs) is widely used to determine compensatory mechanisms of people with anterior cruciate ligament deficiency (ACLD). However, the practicality of the measurements is subject to their reliability during different trials. This study aims to determine the reliability and repeatability of knee joint rotations and GRFs in people with ACLD during stair ascent. Eight participants with unilateral ACL-deficient knees performed five trials of stair ascent with each leg. The movements were captured by VICON motion analysis system, and GRF components were recorded using force plate. Three-dimensional tibiofemoral joint rotations were calculated. Intraclass correlation coefficient (ICC), standard error of measurement (SEM) and coefficient of multiple correlation (CMC) were calculated ACL-deficient legs showed lower absolute reliability during swing ([Formula: see text]–6.4) than stance phase ([Formula: see text]–2.2) for knee joint rotations. Moderate to high average measure ICCs (0.59–0.98), relative reliability, were achieved for injured and uninjured sides. The results also demonstrated high repeatability for the knee joint rotation ([Formula: see text]–0.97) and GRF ([Formula: see text]–0.99). The outcomes of this study confirmed the consistency and repeatability of the knee joint rotations and GRFs in ACL-deficient subjects. Additionally, ACL-deficient legs exhibited similar levels of reliability and repeatability compared to contralateral legs.

Individual Gestalt Is Unreliable for the Evaluation of Quality in Medical Education Blogs: A METRIQ Study

Open educational resources such as blogs are increasingly used for medical education. Gestalt is generally the evaluation method used for these resources; however, little information has been published on it. We aim to evaluate the reliability of gestalt in the assessment of emergency medicine blogs. We identified 60 English-language emergency medicine Web sites that posted clinically oriented blogs between January 1, 2016, and February 24, 2016. Ten Web sites were selected with a random-number generator. Medical students, emergency medicine residents, and emergency medicine attending physicians evaluated the 2 most recent clinical blog posts from each site for quality, using a 7-point Likert scale. The mean gestalt scores of each blog post were compared between groups with Pearson's correlations. Single and average measure intraclass correlation coefficients were calculated within groups. A generalizability study evaluated variance within gestalt and a decision study calculated the number of raters required to reliably (>0.8) estimate quality. One hundred twenty-one medical students, 88 residents, and 100 attending physicians (93.6% of enrolled participants) evaluated all 20 blog posts. Single-measure intraclass correlation coefficients within groups were fair to poor (0.36 to 0.40). Average-measure intraclass correlation coefficients were more reliable (0.811 to 0.840). Mean gestalt ratings by attending physicians correlated strongly with those by medical students (r=0.92) and residents (r=0.99). The generalizability coefficient was 0.91 for the complete data set. The decision study found that 42 gestalt ratings were required to reliably evaluate quality (>0.8). The mean gestalt quality ratings of blog posts between medical students, residents, and attending physicians correlate strongly, but individual ratings are unreliable. With sufficient raters, mean gestalt ratings provide a community standard for assessment.

The Design, Development, and Reliability Testing of a New Innovative Device to Measure Ankle Joint Dorsiflexion.

In clinical and research settings, ankle joint dorsiflexion needs to be reliably measured. Dorsiflexion is often measured by goniometry, but the intrarater and interrater reliability of this technique have been reported to be poor. Many devices to measure dorsiflexion have been developed for clinical and research use. An evaluation of 12 current tools showed that none met all of the desirable criteria. The purpose of this study was to design and develop a device that rates highly in all of the criteria and that can be proved to be highly reliable. While supine on a treatment table, 14 participants had a foot placed in the Charles device and ankle joint dorsiflexion measured and recorded three times with a digital inclinometer. The mean of the three readings was determined to be the ankle joint dorsiflexion. The analysis used was intraclass correlation coefficient (ICC). There was very little difference in ICC single or average measures between left and right feet, so data were pooled (N = 28). The single-measure ICC was 0.998 (95% confidence interval, 0.996-0.998). The average-measure ICC was 0.998 (95% confidence interval, 0.995-0.999). Limits of agreement for the average measure were also very good: -1.30° to 1.65°. The Charles device meets all of the desirable criteria and has many innovative features, increasing its appropriateness for clinical and research applications. It has a suitable design for measuring dorsiflexion and high intrarater and interrater reliability.

Evaluating a natural horsemanship program in relation to the ISES first principles of horse training

The ISES training principles provide an excellent starting point for professionals and horse owners. Currently, there does not seem to be an accepted protocol for evaluating horse training programs against the ISES principles. We suggest an approach to this, using Parelli Natural Horsemanship as our subject for evaluation. This initial pilot study (single-subject / n=1), trials two analytical methods, as applied to the current, video-based teaching materials from Parelli (latest DVD set, published and commercially available from 2015, supplied by Parelli for use in this study). The two methods used were: (i) ethology-based video observation / logging and (ii) discourse analysis of the language used to teach. The ethology-based approach uses an ethogram, which lists the behavioural characteristics of a human trainer adhering to the ISES principles. Computer-based ‘continuous sampling’ of Parelli video clips was used to log the frequencies of ISES principles. Inter Observer Reliability of the analysis to date was assessed using a two-way, mixed, absolute agreement, average-measures ICC (Intra Class Correlation). This evaluated observer agreement in the frequency count ratings for the ISES principles. Discourse analysis is a qualitative research methodology, applied across many domains including politics and health. Discourse analysis allows us to study transcripts of horse training materials, codifying the words, phrases and linguistic structures. Understanding the context within which training language is used, and its meaning to both the speaker and audience, makes it possible to evaluate compatibility with the ISES principles. Results for the ethology-based observations found all ISES principles present (1-10). High frequency counts for principles 2 & 10. Low counts for principles 5 & 7. Inter Observer Reliability (2 observers) was in the ‘excellent’ range (ICC=0.79). The high ICC value suggests that a minimal amount of measurement error was introduced by the independent observers, and therefore statistical power is not substantially reduced. At this stage (without an ICC value closer to 1.0 or further calibrating observers), increasing the evidence against random effects would require more extensive trials (p=0.16). The interim results from the discourse analysis shows consistent congruence between the Parelli materials and the ISES principles, particularly in the areas of: training according to the horse’s ethology and cognition, using learning theory appropriately, forming consistent habits, avoiding flight responses and ensuring that the horse should always be as calm as possible (1, 2, 7, 9 and 10).

Telemedicine: A Reliable Tool to Assess the Severity of Respiratory Distress in Children.

Remote assessment of respiratory distress using telemedicine enabled audio-video conferencing (TM) is of value for medical decision-making. Our goal was to evaluate the interobserver reliability (IOR) of TM compared with face-to-face (FTF) assessment of respiratory distress in children. A prospective, cohort study was performed in pediatric emergency department from July 2012 to February 2013. Children (aged 0-18 years) who presented with signs of respiratory distress were included in the study. The respiratory score is a 4-item, 12-point scale (respiratory rate [1-3], retractions [0-3], dyspnea [0-3], and wheezing [0-3]) that assesses the severity of a child's respiratory distress. Each child was evaluated by a pair of observers from a pool of 25 observers. The first observer evaluated the patient FTF, and the second observer simultaneously and independently evaluated remotely via TM. The overall respiratory distress severity is based on the respiratory scale and reported as nonsevere (≤8) and severe (≥9) respiratory distress. The IOR reliability between FTF and TM assessment was measured using a 2-way mixed model, absolute agreement and average measure intraclass correlation coefficient (ICC). Forty-eight patients and 135 paired observations were recorded. IOR between the FTF and TM groups for total respiratory score had an ICC of 0.95 (confidence interval 0.93-0.96) and for subscores, the ICC range was as follows: respiratory rate = 0.92, retractions = 0.85, dyspnea = 0.94, and wheezing = 0.77. TM is a reliable tool to assess the severity of respiratory distress in children.