Intra-rater Reliability Scores Research Articles

Reliability analysis of WBCT-derived 3D models for comparing preoperative and postoperative alignment in total ankle arthroplasty

BackgroundTraditional imaging techniques for total ankle arthroplasty (TAA) evaluation are limited by rotational bias and bone superimposition, highlighting the necessity for more precise assessment methods. The advent of weight-bearing computed tomography (WBCT) generated 3D enhance the visualization of foot and ankle alignment, offering unmatched detail. This study aims to assess the accuracy of preoperative and postoperative measurements in TAA across all three planes using WBCT-generated 3D models. We hypothesize that these models can be reliably used to compare preoperative and postoperative alignment. MethodsFor 81 patients undergoing TAA, preoperative and postoperative WBCT models were created. Measurements included five coronal angles: Alpha, Tibiotalar Surface Angle (TSA), Talar Tilt Angle (TT), Salzmann's 20 degrees Angle (SA), and Talocalcaneal Angle (TCA); three sagittal angles: Beta, Gamma, and Tibiotalar Ratio (TTR); and one axial angle: The Posterior Talar Rotational Angle (PTARA). Two raters evaluated these before and after surgery in two separate sessions. The study then compared preoperative to postoperative measurements, calculating inter-rater and intra-rater reliability. ResultsSignificant changes were observed in three coronal angles (TSA, TT, and SA) and two sagittal angles (Beta and Gamma), with P-values of 0.2, 0.007, 0.019, <0.001, and <0.001, respectively. No significant changes were noted in Alpha, TCA, TTR, and PTARA, with P-values of 0.2, 0.9, 0.2, and 0.6, respectively. Intra-rater and inter-rater reliability scores ranged from 0.885 to 0.97, indicating good to excellent interclass correlation across all planes, both pre-and postoperatively. ConclusionWBCT-generated 3D modeling and image analysis software have enabled a detailed comparison between preoperative alignment and postoperative TAA positioning across coronal, sagittal, and axial planes, revealing significant adjustments in coronal and sagittal alignments. The high reliability and reproducibility of these measurements affirm their value in preoperative planning in improving the accuracy of surgical interventions. Level of evidenceLevel III of evidence

Spectral features related to the auditory perception of twang-like voices

Background To the best of our knowledge, studies on the relationship between spectral energy distribution and the degree of perceived twang-like voices are still sparse. Through an auditory-perceptual test we aimed to explore the spectral features that may relate with the auditory-perception of twang-like voices. Methods Ten judges who were blind to the test’s tasks and stimuli rated the amount of twang perceived on seventy-six audio samples. The stimuli consisted of twenty voices recorded from eight CCM singers who sustained the vowel [a:] in different pitches, with and without a twang-like voice. Also, forty filtered and sixteen synthesized-manipulated stimuli were included. Results and conclusions Based on the intra-rater reliability scores, four judges were identified as suitable to be included in the analyses. Results showed that the frequency of F1 and F2 correlated strongly with the auditory-perception of twang-like voices (0.90 and 0.74, respectively), whereas F3 showed a moderate negative correlation (–0.52). The frequency difference between F1 and F3 showed a strong negative correlation (–0.82). The mean energy between 1–2 kHz and 2–3 kHz correlated moderately (0.51 and 0.42, respectively). The frequency of F4 and F5, and the energy above 3 kHz showed weak correlations. Since the spectral changes under 2 kHz have been associated with the jaw, lips, and tongue adjustments (i.e. vowel articulation) and a higher vertical laryngeal position might affect the frequency of all formants (including F1 and F2), our results suggest that vowel articulation and the laryngeal height may be relevant when performing twang-like voices.

Reliability of landmark identification for analysis of the temporomandibular joint in real-time MRI

BackgroundReal-time magnetic resonance imaging (rtMRI) is essential for diagnosing and comprehending temporomandibular joint (TMJ) movements. Current methods for tracking and analysis require manual landmark placement on each acquisition frame. Therefore, our study aimed to assess the inter- and intra-rater reliability of placing cephalometric landmarks in frames from a dynamic real-time TMJ MRI.Material and methodsFour real-time MRIs of the right TMJ were taken during mandibular movement at ten frames per second. Seven dentists identified ten landmarks on two frames (intercuspal position—ICP—and maximum mouth opening—MMO) twice at a two-week interval, yielding 112 tracings. Six typical cephalometric measurements (angles and distances) were derived from these landmarks. The reliabilities of landmarks and measurements were evaluated using distance-based (dbICC), linear mixed effect model intraclass correlation (lmeICC), and standard ICC.ResultsThe average inter-rater reliability for the landmarks stood at 0.92 (dbICC) and 0.93 (lmeICC). The intra-rater reliability scores were 0.97 and 0.98. Over 80% of the landmarks showed an ICC greater than 0.98 (inter-rater) and over 0.99 (intra-rater). The lowest landmark ICC was observed for the orbitale and the oblique ridge of the mandibular ramus. However, the cephalometric angle and distance measurements derived from these landmarks showed only moderate to good reliability, whereas the reliability in the frames with ICP was better than those with MMO. Measurements performed in the ICP frame were more reliable than measurements in the MMO frame.ConclusionWhile dentists reliably localize isolated landmarks in real-time MRIs, the cephalometric measurements derived from them remain inconsistent. The better results in ICP than MMO are probably due to a more familiar jaw position. The higher error rate of the TMJ measurements in MMO could be associated with a lack of training in real-time MRI analysis in dentistry.

Analysis of the concurrent validity and reliability of five common clinical goniometric devices

Measurement errors play an important role in the development of goniometric equipment, devices used to measure range of motion. Reasonable validity and reliability are critical for both the device and examiner before and after to testing in human subjects. The objective is to evaluate the concurrent validity and reliability of five different clinical goniometric devices for the purpose of establishing an acceptable measurement error margin for a novel device. We explored the validity and inter- and intrarater reliability scores of five goniometric devices namely (i) the universal goniometer (UG), a two-armed hand-held goniometer, (ii) the inclinometer (IC), featuring a single base, fluid level, and gravity-weighted inclinometer, (iii) the digital inclinometer (DI), functioning as both a DI and dynamometer, (iv) the smartphone application (SA), employing gyroscope-based technology within a smartphone platform application and (v) the modified inclinometer (MI), a gravity pendulum-based inclinometer equipped with a specialized fixing apparatus. Measurements were obtained at 12 standard angles and 8 human shoulder flexion angles ranging from 0° to 180°. Over two testing sessions, 120 standardized angle measurements and 160 shoulder angle measurements from 20 shoulders were repetitively taken by three examiners for each device. The intraclass correlation coefficient (ICC), standard error of measurement (SEM), and minimal detectable change (MDC) were calculated to assess reliability and validity. Concurrent validity was also evaluated through the execution of the 95% limit of agreement (95% LOA) and Bland–Altman plots, with comparisons made to the UG. The concurrent validity for all device pairs was excellent in both study phases (ICC > 0.99, 95% LOA − 4.11° to 4.04° for standard angles, and − 10.98° to 11.36° for human joint angles). Inter- and intrarater reliability scores for standard angles were excellent across all devices (ICC > 0.98, SEM 0.59°–1.75°, MDC 1°–4°), with DI showing superior reliability. For human joint angles, device reliability ranged from moderate to excellent (ICC 0.697–0.975, SEM 1.93°–4.64°, MDC 5°–11° for inter-rater reliability; ICC 0.660–0.996, SEM 0.77°–4.06°, MDC 2°–9° for intra-rater reliability), with SA demonstrating superior reliability. Wider angle measurement however resulted in reduced device reliability. In conclusion, our study demonstrates that it is essential to assess measurement errors independently for standard and human joint angles. The DI is the preferred reference for standard angle testing, while the SA is recommended for human joint angle testing. Separate evaluations across the complete 0°–180° range offer valuable insights.

Evaluating alternatives to dual-energy x-ray absorptiometry for assessing bone quality in patients undergoing spine surgery.

The purpose of this study was to compare and contrast lumbar bone quality and osteoporosis/osteopenia screening results via dual-energy x-ray absorptiometry (DEXA), CT, and MRI. A consecutive series of 426 candidates screened for lumbar disc replacement over a 5-year period beginning in 2018 was reviewed. Patients with a preoperative lumbar spine DEXA scan and a CT and/or MRI scan were included. The primary outcome measures included the bone mineral density (BMD) and osteoporosis or osteopenia classification from DEXA scans, Hounsfield units (HUs) for CT, and vertebral bone quality (VBQ) assessment for MRI. Patients were included if they had a DEXA scan within 1 year of an MRI or CT scan. DEXA BMD scores from composite or level-by-level reports were recorded. Asynchronous MRI and CT measurements were conducted using PACS. Interrater and intrarater reliability scores were generated for both CT and MRI measurements and ranged from 1.000 for MRI L1-4 scans to 0.683 for MRI VBQ. All 3 types of scans were statistically significantly correlated with one another; however, CT was more strongly correlated with the lumbar DEXA value (r = 0.439, p < 0.001). The correlation between MRI VBQ and DEXA was -0.103, (p < 0.045). The CT level-by-level measurements correlate with the corresponding level-by-level DEXA BMD values (correlation ranging from 0.531 to 0.289, p < 0.001 to p = 0.007). CT HU values were more strongly related to osteoporosis/osteopenia classification based on DEXA T-scores than were MRI VBQ values. Receiver operating characteristic analyses found that the area under the curve was 0.817 for CT and 0.539 for MRI. These results demonstrate that CT HUs more closely correlate to DEXA scores than MRI VBQ in this population of patients undergoing surgery for symptomatic disc degeneration. Thus, CT may be an alternative to DEXA for assessing VBQ in this population.

Asymmetric involvement of hands: Psychometric properties of the Turkish version of the Bimanual Fine Motor Function 2.0 classification in children with cerebral palsy

BackgroundManual functions affect more than a half of children with Cerebral palsy (CP). Asymmetric involvement of hands may also affect unilateral and bilateral activities of daily life. The Bimanual Fine Motor Function version 2.0 (BFMF-2.0) is a unique functional classification that categorizes the capacity of each hand (what the child can do) during bimanual functions. PurposeThe aim of this study was to investigate the validity and reliability of the Turkish version of the BFMF-2.0 in children with CP. Study DesignClinical measurement and cross-sectional study. MethodsThe study included 91 children with CP (56 girls, mean age; 7.41 ± 4.23 years [4–18 years]) and their parents. The Manual Ability Classification System (MACS), the Quality of Upper Extremity Skills Test (QUEST), and the Box and Block Test (BBT) were used for construct and concurrent validity. Experienced/inexperienced therapists and parents classified fine motor capacities of the children via live or video-based observation to assess inter-rater reliability. Three weeks later, the children were reclassified for intra-rater reliability. ResultsThe Turkish version of the BFMF-2.0 classification was strongly correlated with the MACS (rho = −0.88, p < 0.001), the QUEST (rho = 0.80, p < 0.001), and the BBT (rho = −0.77, p < 0.001). The inter-rater reliability scores were weak to excellent between the parents and the therapists (via live observation, κw = 0.57) and also between experienced/inexperienced therapists (via live or video-based observation, κw = 0.66–0.79). Intra-rater reliability scores were good to excellent (Intraclass Correlation Coefficient [ICC] = 0.87–0.95). ConclusionsThe Turkish version of the BFMF-2.0 classification is valid and reliable and could be applied by experienced and inexperienced therapists via live or video-based observation and by parents via live observation.

Orthodontic treatment with miniscrew anchorage: Analysis of quality of information on YouTube

The evaluation of the quality of information (QOI) and clarity of information (COI) among oral health-related videos on the video-streaming Web site YouTube is scarce. This study evaluated QOI and COI regarding temporary anchorage devices contained within videos uploaded by dental professionals (DPs) on YouTube. YouTube videos were systematically acquired with 4 search terms. The top 50 videos per search term by the number of views were stored in a YouTube account. A set of inclusion/exclusion criteria were applied, videos were assessed for viewing characteristics, a 4-point scoring system (0-3) was applied to evaluate QOI in 10 predetermined domains, and a 3-point scoring system (0-2) was applied to evaluate COI. Descriptive statistical analyses and intrarater and interrater reliability tests were performed. Strong intrarater and interrater reliability scores were observed. Sixty-three videos from the top 58 most-viewed DPs were viewed 1,395,471 times (range, 414-124,939). Most DPs originated from the United States (20%), and orthodontists (62%) uploaded most of the videos. The mean number of reported domains was 2.03 ± 2.40 (out of 10). The mean overall QOI score per domain was 0.36 ± 0.79 (out of 3). The "Placement of miniscrews" domain scored highest (1.23 ± 0.75). The "Cost of miniscrews placement" domain scored the lowest (0.03 ± 0.25). The mean overall QOI score per DP was 3.59 ± 5.64 (out of 30). The COI in 32 videos was immeasurable, and only 2 avoided using technical words. The QOI related to temporary anchorage devices contained within videos provided by DPs through the YouTube Web site is deficient, particularly in the cost of placement. Orthodontists should be aware of the importance of YouTube as an information resource and ensure that videos related to temporary anchorage devices contain comprehensive and evidence-based information.

Test-Retest, Interrater Reliability, and Minimal Detectable Change of the Dynamic Exertion Test (EXiT) for Concussion.

The Dynamic Exertion Test (EXiT) was developed to inform return-to-play (RTP) decision-making following clinical recovery from sport-related concussion (SRC). The purpose of the current study was to document intrarater and test-retest reliability and minimal detectable change (MDC) scores for physiological [heart rate (HR) and blood pressure], performance (change-of-direction task completion time and errors), and clinical outcomes (endorsed symptoms, perceived exertion) of EXiT, and interrater reliability of performance outcomes. Healthy athletes would exhibit stable physiological responses to the EXiT across visits, demonstrate consistent change-of-direction task completion time between consecutive trials at each visit, and the fastest time (of 2 trials) across visits, and endorse equivocal symptoms and effort across visits. Cross-sectional, test-retest. Level 3. Seventy-nine (female: 34 [43%], 19.6 ± 5.0 years) athletes completed the EXiT at 2 study visits (8.7 ± 4.7 days between visits). Two-way, mixed, intraclass correlation coefficients (ICCs) were used to evaluate intrarater and test-retest reliability. Cronbach's alpha was used to document the internal consistency of symptoms at each visit, and MDC scores were calculated on the physiological, performance, and clinical outcomes. Measured and percentage of age-estimated maximum HR were reliable following EXiT (ICC = 0.579-0.618). Change-of-direction task completion time (MDC range = 0.75-8.70 s) had good-to-excellent test-retest (ICC = 0.703-0.948) and interrater (ICC = 0.932-0.965) reliability. Symptoms had a high internal consistency at visits 1 (α = 0.894) and 2 (α = 0.805) and were reliable across visits (ICC = 0.588). The current investigation established test-retest reliability in addition to MDC scores of an objective dynamic exercise assessment among healthy adolescent and adult athletes. The EXiT may be an objective approach to inform RTP decision-making following SRC recovery. The EXiT is a clinically feasible exertion-based assessment that can be readily administered in a variety of outpatient clinical settings.

Reliability of functional outcome measures in adults with neurofibromatosis 2.

Objective:To determine inter- and intra-rater reliability of functional performance outcome measures in people with neurofibromatosis 2. To ascertain how closely objective and subjective measures align.Methods:Twenty-nine people with neurofibromatosis 2 were recorded performing the modified clinical test of sensory integration and balance, four square step test and modified nine-hole peg tests. Three raters scored each measure to determine inter-rater reliability. One rater scored the measures a second time to determine intra-rater reliability. Participants also completed a disease-specific quality of life questionnaire and dynamic visual acuity testing.Results:Inter-rater and intra-rater reliability scores (intra-class correlation coefficient) were excellent for all tests (intra-class correlation coefficient r ⩾ 0.9). The four square step test correlated with perceived walking challenges and modified clinical test of sensory integration and balance correlated with perceived balance challenges in a neurofibromatosis 2 quality of life patient report outcome measure.Conclusion:The modified clinical test of sensory integration and balance, four square step test and modified nine-hole peg tests are potentially useful measures for monitoring neurofibromatosis 2.

P38. A novel fusion classification system for the advent of bioactive interbody implants

BACKGROUND CONTEXT Historically, intervertebral fusion and biologically-mediated stability required bone bridging exclusively. The advent of bioactive interbody devices, which allow for appositional integration of bone at the endplate-implant interface, demands an evolution in our radiographic assessment of such fusion processes. Specifically, acknowledgement that endplate bone adhesion to the surface of these implant confers stability, and potentially an effective ‘fusion’, warrants strong consideration within a new classification scheme. PURPOSE We present a novel interbody fusion classification system that addresses inadequacies if traditional grading schemes in order to accommodate the integration capability of bioactive intervertebral cages. STUDY DESIGN/SETTING Classification System Reliability Survey. PATIENT SAMPLE A total of 37 surgeon respondents. OUTCOME MEASURES Intraclass correlation coefficients for inter- and intra-rater reliability assessments. METHODS A novel grading system for intervertebral fusion was developed by the authors (Figure) and included fusion consolidation through and/or around the intervertebral device, as well as the apposition of bone to the surface of the implants (ingrowth/ongrowth). Ten patient cases with 1year CT scans were selected for the survey, including coronal and sagittal reconstruction views which were provided to the respondents. Surveys were administered twice >2weeks apart and with cases in differential order. Intraclass Correlation Coefficient (ICC) was used to calculate inter and intrarater reliability. RESULTS A total of 51 spine surgeons participated, with 37 of them completing surveys on two separate occasions. For the entire cohort, ICC inter-rater reliability (95% CI) average was 0.618 (0.435-0.834; p<0.001), while the intra-rater reliability average was 0.778 (SD=0.211). Four respondents had outlier intra-rater reliability scores averaged 0.259. When removing those 4 respondents (10.8% of the respondents), and analyzing the more consistent 90% of the respondents, ICC inter-rater reliability average was 0.628 (0.445-0.841; p=<0.001), and intra-rater reliability average was 0.831 (SD=0.125). CONCLUSIONS Our novel CT-based interbody fusion classification system demonstrated substantial inter-rater reliability, and excellent intra-rater reliability when applied to a spectrum of real cases. Future assessment of interbody fusions should consider utilization of this new system in order to address the important contribution of appositional ongrowth/ingrowth, along with traditional bone healing through and around devices. FDA DEVICE/DRUG STATUS This abstract does not discuss or include any applicable devices or drugs. Historically, intervertebral fusion and biologically-mediated stability required bone bridging exclusively. The advent of bioactive interbody devices, which allow for appositional integration of bone at the endplate-implant interface, demands an evolution in our radiographic assessment of such fusion processes. Specifically, acknowledgement that endplate bone adhesion to the surface of these implant confers stability, and potentially an effective ‘fusion’, warrants strong consideration within a new classification scheme. We present a novel interbody fusion classification system that addresses inadequacies if traditional grading schemes in order to accommodate the integration capability of bioactive intervertebral cages. Classification System Reliability Survey. A total of 37 surgeon respondents. Intraclass correlation coefficients for inter- and intra-rater reliability assessments. A novel grading system for intervertebral fusion was developed by the authors (Figure) and included fusion consolidation through and/or around the intervertebral device, as well as the apposition of bone to the surface of the implants (ingrowth/ongrowth). Ten patient cases with 1year CT scans were selected for the survey, including coronal and sagittal reconstruction views which were provided to the respondents. Surveys were administered twice >2weeks apart and with cases in differential order. Intraclass Correlation Coefficient (ICC) was used to calculate inter and intrarater reliability. A total of 51 spine surgeons participated, with 37 of them completing surveys on two separate occasions. For the entire cohort, ICC inter-rater reliability (95% CI) average was 0.618 (0.435-0.834; p<0.001), while the intra-rater reliability average was 0.778 (SD=0.211). Four respondents had outlier intra-rater reliability scores averaged 0.259. When removing those 4 respondents (10.8% of the respondents), and analyzing the more consistent 90% of the respondents, ICC inter-rater reliability average was 0.628 (0.445-0.841; p=<0.001), and intra-rater reliability average was 0.831 (SD=0.125). Our novel CT-based interbody fusion classification system demonstrated substantial inter-rater reliability, and excellent intra-rater reliability when applied to a spectrum of real cases. Future assessment of interbody fusions should consider utilization of this new system in order to address the important contribution of appositional ongrowth/ingrowth, along with traditional bone healing through and around devices.

Cutaneous Lupus Erythematosus: Disease Area and Severity Index reliability in clinical evaluation and assessment of cutaneous lupus erythematosus lesions in pediatric cases

Background: Children having systemic lupus erythematosus clinically reflect a management and diagnostic challenge. Cutaneous lupus erythematosus reflects the dermatological signs of SLE categorized into LE-specific and LE-nonspecific features. The Cutaneous Lupus Erythematosus Disease Area and Severity Index (CLASI) and Physician Global Assessment (PGA) are clinical tools categorizing the dermatological disease severity in cases having cutaneous lupus disease Aim of study: Assessment of CLASI scoring reliability and usefulness in pediatric cutaneous lupus. Methodology: A clinical research trial conducted on 100 research study subjects with pediatric cutaneous lupus erythematosus. Cases had an age range of 6–18 years. All research study subjects were assessed using both the CLASI and skin Physician’s Global Assessment clinical tools. Results: CLASI activity had excellent inter-rater and intra-rater reliability scores among dermatologists and rheumatologists (ICC = 0.965, 0.993, 0.924, 0.975 consecutively), CLASI damage had good interrater and intra rater scores among dermatologists (ICC = 0.810, 0.893 consecutively) on the other hand among rheumatologists it had poor interrater score (ICC =0.397) and good intra rater score (ICC =0.786). PGA activity had good inter rater reliability (ICC=0.830) and excellent intra rater variability (0.912) among dermatologists and among rheumatologists had fair inter rater and intra rater reliability (ICC=0.754, 0.784 consecutively) PGA damage had moderate inter rater reliability (ICC=0.593, 0.684) and fair intra rater reliability (ICC=0.783, 0.756) among dermatologists and rheumatologists consecutively.

Orthodontic retention and retainers: Quality of information provided by dental professionals on YouTube

The evaluation of the quality of information (QOI) contained within oral health-related videos on the video-streaming Web site YouTube has been limited. This study aimed to determine the QOI regarding orthodontic retention and retainers contained within videos uploaded by dental professionals (DPs) to YouTube. YouTube was systematically searched using 4 terms. The top 50 videos per search term by "number of times viewed" were stored in a specially created account. Following the application of inclusion/exclusion criteria, the videos were assessed for viewing characteristics, and a 4-point scoring system (0-3) was applied to evaluate QOI in 10 predetermined domains. Descriptive statistical analyses, intrarater, and interrater reliability tests were performed. Strong intrarater and interrater reliability scores were observed. Sixty-two videos from the top 50 most-viewed DPs were viewed a total of 1,367,810 times (range 77- 529,543). Most DPs originated from the United States (68%), and orthodontists (88%) uploaded most videos. The mean number (standard deviation [SD]) of reported domains was 5.5 out of 10 (SD, 2.2). The mean overall QOI score per domain was 1.14 out of 3 (SD, 0.64). The "importance of retention" domain scored highest (2.18; SD, 1.2). "The need for retention reviews" domain scored the lowest (0.3; SD, 0.65). The mean overall QOI score per DP was 11.72 out of 30.00 (SD, 5.85). The QOI related to orthodontic retention and retainers contained within videos provided by DPs through the YouTube Web site is deficient, particularly the need for retainer review. Orthodontists should be aware of the importance of YouTube as an information resource and ensure that videos related to orthodontic retention and retainers contain relevant, high-quality, and evidence-based information.

Reliability testing of the Matching Assessment using Photographs of Scars App.

The Matching Assessment using Photographs with Scars assessment tool, published in 2005, enables accurate relocation and reassessment of scars. While used in Australia, uptake has been hampered by its paper manual format. With electronic records and increasing use of smart devices in health, here we report the development of the Matching Assessment using Photographs with Scars manual into a mobile application format: ClinMAPSTM Pro. At the time of development, no other digital scar assessment applications were available. For clinical validation, the digitized Matching Assessment using Photographs with Scars module within ClinMAPSTM Pro was used for intra- and inter-rater reliability testing. Convenience sampling was utilized to recruit burns patients representing 44 scars, based on predetermined power calculations. Three therapists, one experienced and two novice, acted as the assessors. Each therapist assessed preselected scars with the digitized Matching Assessment using Photographs with Scars. Re-assessment of the same scar sites occurred 3 to 7 days later. Inter-rater reliability testing scores of the new electronic assessment application showed fair to moderate agreement (combined Fleiss Kappa = 0.38-0.49, P < .0001). Intra-rater reliability scores between initial and repeat measures showed moderate to almost perfect agreement (Cohen's Kappa = 0.43-0.90, P < .0001). It was noted that for an experienced rater, intra-rater agreement demonstrated substantial to almost perfect agreement. Intraclass correlation coefficients calculated to allow for comparison between other published scar assessment tools demonstrated excellent reliability for all scar assessment parameters for both intra- (=0.76-0.91) and inter- (=0.76-0.98) reliability. Notably, reliability testing results confirm its intra- and inter-rater reliability. Our findings validate this novel concept for digitization of the previously paper-based scar assessment manual and prove that the Matching Assessment using Photographs with Scars within the ClinMAPSTM Pro application is now readily available for clinicians and researchers internationally.

Development and preliminary validation of the Coach Interpersonal Style Observational System

ObjectivesThe aim of this study was to design and develop a multidimensional observational coding system (Coach Interpersonal Style Observational System) to assess coaches' interpersonal style adopted during training sessions and matches.Equipment and methodsContent validity was established by 11 experts (with proven knowledge and experience in sport science, motivational psychology, and coaching) using qualitative (e.g. ease of comprehension, utility, and adequacy of examples) and quantitative (category scores) approaches. Coders were trained to test inter- and intra-rater reliability.ResultsQualitative analyses revealed global positive assessment. Experts' category scores were very high. After coder training, inter- and intra-rater reliability scores were acceptable. Discriminant validity was tested through the pilot application to eight soccer coaches. Overall, the results support the preliminary reliability and validity of an instrument to analyze interpersonal coach behavior.

Reliability of Oronasal Fistula Classification.

Oronasal fistula is an important complication of cleft palate repair that is frequently used to evaluate surgical quality, yet reliability of fistula classification has never been examined. The objective of this study was to determine the reliability of oronasal fistula classification both within individual surgeons and between multiple surgeons. Using intraoral photographs of children with repaired cleft palate, surgeons rated the location of palatal fistulae using the Pittsburgh Fistula Classification System. Intrarater and interrater reliability scores were calculated for each region of the palate. Eight cleft surgeons rated photographs obtained from 29 children. Within individual surgeons reliability for each region of the Pittsburgh classification ranged from moderate to almost perfect (κ = .60-.96). By contrast, reliability between surgeons was lower, ranging from fair to substantial (κ = .23-.70). Between-surgeon reliability was lowest for the junction of the soft and hard palates (κ = .23). Within-surgeon and between-surgeon reliability were almost perfect for the more general classification of fistula in the secondary palate (κ = .95 and κ = .83, respectively). This is the first reliability study of fistula classification. We show that the Pittsburgh Fistula Classification System is reliable when used by an individual surgeon, but less reliable when used among multiple surgeons. Comparisons of fistula occurrence among surgeons may be subject to less bias if they use the more general classification of "presence or absence of fistula of the secondary palate" rather than the Pittsburgh Fistula Classification System.

An Intercenter Comparison of Nasolabial Appearance Including a Center Using Nasoalveolar Molding

To compare nasolabial appearance outcomes of patients with complete unilateral cleft lip and palate (CUCLP) in preadolescence from 4 cleft centers including a center using nasoalveolar molding (NAM) and primary nasal reconstruction. Retrospective cohort study. Four cleft centers in North America. 135 subjects with repaired CUCLP. Frontal and profile facial pictures were assessed using the Asher-McDade rating scale. Intra- and interrater reliability were tested using weighted Kappa statistics. Median scores by center were compared with Kruskal-Wallis statistics. Intrarater reliability scores were moderate to good. Interrater reliability scores were moderate. Significant differences ( P < .05) among centers were found. For nasal form, center G (median = 2.83) had better scores than centers C and D (C median = 3.33, D median = 3.17). For nose symmetry, center G had better scores (median = 2.33) than all other centers (B median = 2.67, C median = 2.83, D median = 2.83). For vermillion border, center G had better scores (median = 2.58) than centers B and C (B median = 3.17, C median = 3.17). For nasolabial profile, center G (median score = 2.67) had better scores than center C (median = 3.00). For total nasolabial score, center G (median = 2.67) had better scores than all other centers (B median = 2.83, C median = 3, D median = 2.83). The protocol followed by center G, the only center that performed NAM and primary nasal reconstruction, produced better results in all categories when compared to center C, the only center that did not perform presurgical orthopedics or lip/nose revisions. When compared to centers that performed traditional presurgical orthopedics and surgical revisions (B and D), center G was not consistently better in all categories. As with other uncontrolled, retrospective intercenter studies, it is not possible to attribute the outcomes to a specific protocol component.

Reliability of functional outcome measures in adults with neurofibromatosis 1.

Objectives:To determine intra-rater and inter-rater reliability of functional outcome measures in adults with neurofibromatosis 1 and to ascertain how closely objective and subjective measures align.Methods:A total of 49 ambulant adults with neurofibromatosis 1 aged 16 years and over were included in this observational study: median age 31 years (range: 16–66 years), 29 females, 20 males. Participants were video-recorded or photographed performing four functional outcome measures. Four raters from the neurofibromatosis centre multi-disciplinary team independently scored the measures to determine inter-rater reliability. One rater scored the measures a second time on a separate occasion to determine intra-rater reliability. The measures evaluated were the functional reach, timed up and go, 10 m walk and a modified nine-hole peg tests. Participants also completed a disease-specific quality-of-life questionnaire.Results:Inter-rater reliability and intra-rater reliability scores (intra-class coefficient) were similar for each outcome measure. Excellent rater agreement (intra-class coefficient, r ⩾ 0.9) was found for the functional reach, timed up and go and the 10 m walk tests. Rater agreement was good for the modified nine-hole peg test: intra-class coefficient r = 0.75 for intra-rater reliability and 0.76 for inter-rater reliability. The timed up and go and the 10 m walk tests correlated highly with perceived mobility challenges in the quality-of-life questionnaire.Conclusion:The functional reach, timed up and go and 10 m walk tests are potentially useful outcome measures for monitoring neurofibromatosis 1 treatment and will be assessed in multi-centre and longitudinal studies.

Impact of Software Selection and ImageJ Tutorial Corrigendum on Skeletal Muscle Measures at the Third Lumbar Vertebra on Computed Tomography Scans in Clinical Populations

There is growing interest in computed tomography (CT) measures of skeletal muscle cross-sectional area (CSA) for nutrition assessment. Multiple software programs are available, but little work has been done comparing programs. We aimed to determine if CT-derived measures of skeletal muscle CSA at the level of the L3 are influenced by the software program used. We also demonstrate the importance of the ImageJ corrigendum published in this journal. Two software programs, National Institutes of Health ImageJ and Tomovision sliceOmatic, were compared. ImageJ measures were obtained using both the original tutorial and corrigendum instructions. Skeletal muscle CSA at the level of the L3 was measured in advanced heart failure and head and neck cancer populations by 3 different investigators. Intraclass correlation coefficients were used to calculate intrarater and interrater reliability. Bland-Altman analysis was used to assess agreement. Both software programs yielded excellent intrarater and interrater reliability scores (intraclass correlation coefficients, 0.985-1.000). The overall mean difference (ImageJ tutorial with corrigendum - sliceOmatic) for the entire sample (N = 51) was found to be 1.53 cm2 (95% CI, 0.59-2.47 cm2 ). The overall mean difference (ImageJ corrected - original) for the entire sample (N = 51) was found to be -11.35 cm2 (95% CI, -12.75 to -9.95 cm2 ). Measures of skeletal muscle CSA at the L3 were found to be ∼1.53 cm2 higher with ImageJ than sliceOmatic. This difference was not found to affect interpretation against a published cut point. The importance of accounting for the ImageJ tutorial corrigendum was shown to be clinically significant when applied to published cut points.

Developing a contributing factor classification scheme for Rasmussen's AcciMap: Reliability and validity evaluation

One factor potentially limiting the uptake of Rasmussen's (1997) Accimap method by practitioners is the lack of a contributing factor classification scheme to guide accident analyses. This article evaluates the intra- and inter-rater reliability and criterion-referenced validity of a classification scheme developed to support the use of Accimap by led outdoor activity (LOA) practitioners. The classification scheme has two levels: the system level describes the actors, artefacts and activity context in terms of 14 codes; the descriptor level breaks the system level codes down into 107 specific contributing factors. The study involved 11 LOA practitioners using the scheme on two separate occasions to code a pre-determined list of contributing factors identified from four incident reports. Criterion-referenced validity was assessed by comparing the codes selected by LOA practitioners to those selected by the method creators. Mean intra-rater reliability scores at the system (M = 83.6%) and descriptor (M = 74%) levels were acceptable. Mean inter-rater reliability scores were not consistently acceptable for both coding attempts at the system level (MT1 = 68.8%; MT2 = 73.9%), and were poor at the descriptor level (MT1 = 58.5%; MT2 = 64.1%). Mean criterion referenced validity scores at the system level were acceptable (MT1 = 73.9%; MT2 = 75.3%). However, they were not consistently acceptable at the descriptor level (MT1 = 67.6%; MT2 = 70.8%). Overall, the results indicate that the classification scheme does not currently satisfy reliability and validity requirements, and that further work is required. The implications for the design and development of contributing factors classification schemes are discussed.

Developing a standardized measurement of alcohol intoxication

IntroductionWe assessed multiple examinations and assessment tools to develop a standardized measurement of alcohol intoxication to aid medical decision making in the Emergency Department. MethodsVolunteers underwent an alcohol challenge. Pre- and post-alcohol challenge, subjects were videotaped performing three standardized clinical examinations: (1) Standardized Field Sobriety Test (SFST) examination, (2) Hack's Impairment Index (HII) examination, and (3) Cincinnati Intoxication Examination (CIE). Emergency clinicians evaluated the level of intoxication using five standardized assessment tools in a blinded and randomized fashion: (1) SFST assessment tool (range 0–18), (2) HII assessment tool (range 0–1), (3) St. Elizabeth Alcohol Intoxication Scale (STE, range 0–17), (4) a Visual Analog Scale (VAS, range 0–100), and (5) a Binary Intoxication Question (BIQ). Construct validity was assessed along with inter- and intra-rater reliability. ResultsMedian scores pre- and post-alcohol challenge were: SFST 6 (interquartile range 5) and 11 (3), respectively; HII 0 (0.05), 0.1 (0.1); STE 0 (1), 1 (2); VAS 10 (22), 33 (31). For BIQ, 59% and 91% indicated intoxication, respectively. Inter-rater reliability scores were: SFST 0.71 (95% confidence interval 0.48–0.86) to 0.93 (0.88–0.97) depending on examination component; HII 0.90 (0.82–0.95); STE 0.86 (0.75–0.93); VAS 0.92 (0.88–0.94); BIQ 0.3. Intra-rater reliability scores were: SFST 0.74 (0.64–0.82) to 0.87 (0.81–0.91); HII 0.85 (0.79–0.90); STE 0.78 (0.68–0.85); VAS 0.82 (0.74–0.87); BIQ 0.71. VAS reliability was best when paired with the HII and SFST examinations. ConclusionsHII examination, paired with either a VAS or HII assessment tool, yielded valid and reliable measurements of alcohol intoxication.