Intra-and Inter-rater Reliability Research Articles

Reference Standard of Median Nerve Conduction Study in Korea.

To establish the reference standard of the median nerve conduction study (NCS) in Korea. A total of 648 median motor and 602 median sensory NCSs from 349 Korean healthy volunteers were tested and analyzed prospectively. Equipment calibration, assessment of intraand inter-rater reliability, and the NCSs per se were conducted according to a predetermined protocol. A reference standard was established from uncertainty components for the following parameters: the onset and peak latencies; the baseline-to-peak and peak-to-peak amplitudes; the area and duration of the negative wave; and the nerve conduction velocity. The effects of sex, age and stimulation intensity were analyzed. Each measured value of 648 median motor and 602 median sensory nerves were obtained and presented with both mean and expanded uncertainties, as well as mean and standard deviations. The cut-off values with expanded uncertainty were determined for different age and sex groups. After adjusting for anthropometric covariates, all parameters except duration were affected by age, and sex appeared to influence both duration and area. While stimulation intensity significantly affected some parameters including latencies, the effect sizes were negligible. We propose the median NCS reference standard using the largest Korean dataset ever available. The use of the traceable and reliable reference standard is anticipated to promote more accurate and dependable diagnosis and appropriate management of median neuropathies in Korea.

Intra-rater and Inter-rater Reliability of the KangaTech (KT360) Fixed Frame Dynamometry System During Maximal Isometric Strength Measurements of the Knee Flexors.

Fixed-frame dynamometry systems are used worldwide to assess isometric strength in both general and athletic populations. There is currently a paucity of published work where reliability estimates for fixed-frame dynamometry systems have been estimated. The aim of this study was to determine the inter-and intra-rater reliability of the KangaTech (KT360) fixed frame dynamometry system when measuring maximal isometric strength of the knee flexor muscles. Inter and intra-rater reliability single cohort study. Twenty healthy university-level athletes (age= 21.65 ± 3 years, weight= 74.465 ± 30kg, height= 170.1 ± 7.0cm) took part in two testing sessions where two raters collected data during a 90° hip and knee flexion protocol. Participants performed each test twice, building to a maximal isometric contraction holding over a 5 second period with 30 second rest between sets. Data were checked for normality using a Shapiro-Wilk test. Intraclass correlation coefficient (ICC), standard error of measurement (SEM) and minimal detectable change (MDC) were calculated. Finally, a Bland-Altman analysis was used to determine the levels of agreement for intra-and inter-rater measurements. High levels of agreement were demonstrated between left and right knee flexion as 95% of the differences were less than two standard deviations away from the mean. 'Almost perfect' intraclass correlation coefficient (ICC) values were demonstrated (Knee flexion: Inter-rater: Left, 0.99; Right, 0.99; Intra-rater: Left, 0.99; Right:0.99). Standard error of measurement (SEM) for inter-and intra-rater strength ranged from 0.26-0.69 kg, SEM% ranged from 1.34-2.71% and minimal detectable change (MDC) ranged from 1.14-2.31kg. Overall, high level of inter-and intra-rater reliability were demonstrated when testing maximal isometric knee flexion. Therefore, the KT360 fixed frame dynamometry system may be considered a viable tool for measuring maximal isometric contraction of the knee flexors when repeat measures are required in clinical settings. 3b.

Subclassification of the Koos grade 2 vestibular schwannoma into 2a and 2b for individualized patient care: A validity and reliability study

ObjectiveVestibular schwannoma (VS) growth of ≥2 mm during serial MRI observation, irrespective of size, is the benchmark for treatment initiation in almost all centers. Although the probability of less optimal outcomes significantly increases in VS closer to the brainstem, early intervention does not improve long-term quality of life. Moving beyond the recommendation of definitive treatment for all VS after detected growth, we subclassified Koos 2 tumors based on extrameatal extension and relation to the brainstem. The aim of the current study was to evaluate the Koos 2 subclassification’s validity and the inter-and intra-rater reliability of the entire Koos classification. MethodsSix experts, including neurosurgeons, otorhinolaryngologists and radiologists from two tertiary referral centers, classified 43 VS MRI scans. Validity of the Koos 2 subclassification was evaluated by the percentage agreement against the multidisciplinary skull base tumor board management advice. Inter- and intra-rater reliability were calculated using the intraclass correlation coefficient (ICC). ResultsValidity was almost perfect in Koos 2a VSs with a 100% agreement and 87.5% agreement for Koos 2b. Inter-rater reliability for all Koos grades was significantly excellent (ICC 0.91; 95%CI 0.866 to 0.944, p= <0.001). Five raters had an excellent intra-rater reliability (ICC > 0.90; p= <0.01) and one rater had a good intra-rater reliability (ICC 0.88; 95% CI 0.742 to 0.949). ConclusionsAlthough multiple factors influence decision-making, the classification of Koos 2a and 2b with excellent inter- and intra-rater reliability, can aid in recommending treatment initiation, moving beyond detected tumor growth, aiming to optimize patient centered care.

Reliability of a standardized tool for evaluating severity of cellulite in the female posterior thigh.

This study evaluated the reliability of a standardized grading system for scoring the overall severity of cellulite on the posterior thigh. The study evaluated inter-rater and intra-rater (test/re-test) reliability of the method and engaged in an iterative process to develop a reliable method to evaluate changes in the appearance of cellulite. There were two stages in the validation process. The first stage was an open process without evaluator training. The second stage was a more controlled process with training given and moderator involvement to review grade selections. In the first stage, inter-rater reliability was examined across five evaluators who were asked to evaluate 24 photographs (right thighs) based on a cellulite graded severity chart. During the second stage, the same photographs were examined by paired evaluators who had received additional training. Scores were independently moderated by a third person. The inter-rater reliability and intra-rater reliability over a 4-week interval were evaluated using intraclass correlation coefficients (ICCs). Twenty-four female participants (18-51 years, mean 31.68 ± 9.03 years) with a mean BMI of 29.04 ± 6.52 participated in the trial. Five female evaluators completed the initial evaluations. In stage 1, the inter-rater reliability (ICC2,5 ) was 0.838 (95%CI:0.700-0.922) and test/retest ICC3,1 values ranged from 0.360-0.990. In stage 2, the inter-rater reliability for 2 evaluators improved to 0.978 (95%CI:0.948-0.991), and the test/retest reliability of the moderated scoring method improved to 0.993 (95%CI:0.983-0.997). The iterative process developed a simple and reliable method of rating cellulite severity, with excellent inter-and intra-rater reliability, based on evaluating images of cellulite against a standard set of graded severity images. A reliable method of assessing cellulite severity is essential for undertaking future clinical trials to evaluate cellulite treatments.

Registration of Bony Landmarks and Soft Tissue Laxity during Robotic Total Knee Arthroplasty is Highly Reproducible.

There is growing interest and enthusiasm for robotic total knee arthroplasty (TKA). Many robotic systems require registration of bony landmarks as well as a dynamic soft tissue evaluation to plan femoral and tibial resections. Variability in this user-driven registration can introduce error and undermine the purported precision and accuracy offered by robotics. The purpose of this study was to evaluate inter- and intrarater reliability in robotic registration with a new robotic system (ROSA®; Zimmer-Biomet, Warsaw, IN). Two unpaired cadaveric knee specimens were exposed, and optical arrays were placed into the femur and tibia. Three separate evaluators conducted repeated trials of anatomic registration and assessment of soft tissue laxity, as well as coronal alignment, sagittal alignment, femoral size, and maximum opening in the medial and lateral compartments in both flexion and extension. Repeated trials were conducted using these specimens with and without preoperative imaging for landmarking (image-based and image-free workflows). An Intraclass Correlation Coefficient (ICC) was calculated for each observer and across observers to determine intra-and interrater reliability, respectively, in robotic registration. There was good to excellent reliability for all conditions, and all correlation coefficients were >0.767. On average, ICCs for intrarater reliability were excellent for Doctor 1 (0.952), Doctor 2 (0.975), and Doctor 3 (0.925). On average, the ICCs for interrater reliability were excellent for both the "Registration + Gap Assessment" condition (0.961) and the "Gap Assessment" condition (0.994). Our results show a high repeatability of registration of anatomic landmarks and gap assessment among observers using this robotic system for both image-based and image-free software.

A new classification for combined greater tuberosity fracture and anterior shoulder dislocation: A study of fracture configurations and displacement after reduction.

The aim of this study was to propose a new classification of combined greater tuberosity (GT) fractures and anterior shoulder dislocation and studied the degree of displacement, functional outcomes, and need for additional surgery after reduction. A cross-sectional study was conducted. We evaluated radiographs of patients treated for combined GT fractures and anterior shoulder dislocation. Three morphologies were proposed; type 1 (a small avulsion), type 2 (GT fractures without articular head involvement), and type 3 (GT associated with articular head fractures). Two orthopedic surgeons independently measured all radiographs and classified fractures into three types. Patients were interviewed by telephone to assess functional outcomes (the simple shoulder test (SST) and EQ-5D-5L), and additional shoulder surgery was also performed. There were 52 eligible patients; 32 were male (61.5%) and the mean age was 57.3 · 17.1 years. Most cases were low-energy injuries (61.5%). Of all the cases, 32.7% were type I, 59.6% type II, and 7.7% type III cases. There were differences in the degree of displacement in each group at pre, post-reduction (both horizontal and vertical planes) and at two weeks post-reduction for HD (p < 0.05). Type III had more displacement than type I at pre- and post-reduction with a P value of less than 0.05. Type III also had higher rates of displacement than type II at post-reduction and at two-week postreduction (vertical plane). The intra and inter-rater reliabilities of measurement (ICC > 0.8) were in good to excellent agreement with the kappa value (>0.9). Three out of 52 cases (5.8%) required an additional surgery after closed reduction. Patients had good functional outcomes (SST score of 8) with an excellent utility index of EQ-5D-5L (0.9). This new classification exhibited good-to-excellent intra-and inter-rater reliabilities, with an ability to determine injury type. Type III seems to be linked to higher risk of fracture displacement and may require additional surgery. Level IV, Diagnostic Study.

The Timed Water Swallow Test (TWST): Normative data on swallowing capacity for healthy people aged 60 years and older

Purpose To provide normative data on swallowing capacity (mL/s) in people older than 60 years using the Timed Water Swallow Test (TWST), stratified by sex. Intra- and inter-rater reliability for swallowing time in TWST was further investigated. Method A total of 165 participants, aged 60 years and above, were included in the study. The time taken to consume 150 mL of tap water and the swallowing capacity (mL/s) was observed in a clinical context using the TWST. Video recordings of the performances were collected for 118 of the 165 participants for inter-rater reliability testing, of which a random set of 25 performances were assessed for intra-rater reliability. Analysis of the agreement between TWST rated by an in-person clinician and rated from a video recording was further evaluated. Result There were significant age and sex effects observed for swallowing capacity. The average reduction in swallowing capacity in the investigated age range (60–92 years) was 8.8 and 7.3 mL/s, for women and men respectively. The results showed high inter-and intra-rater reliability and agreement between the clinical assessment and the video recording. Conclusion The presented measurements indicate a clear effect of age on swallowing capacity and that women may be expected to have lower capacity scores than men when tested using TWST. The provided norms can be used as reference points in the clinical identification of people at risk of dysphagia in the older population.

The Reliability of Two Dimensional Kinematic Assessment of a 45-Degree Side Step Cut

Background:Up to 70% of all anterior cruciate ligament (ACL) injuries occur via a non-contact mechanism involving deceleration and direction change on a planted foot. Clinical screening tools analyzing movement patterns during a cutting task are reliable and valid. The Expanded Cutting Alignment Scoring Tool (E-CAST) is a reliable tool to qualitatively assess trunk and lower extremity (LE) alignment during a 45-degree side step cut (SSC). Previous works found two-dimensional (2D) kinematic analysis more reliable than qualitative assessments when evaluating jumping and squatting movements. However, there is limited evidence on the reliability of 2D assessments to evaluate SSC tasks.Hypothesis/Purpose:To determine if using 2D kinematic analysis with the E-CAST improves the tool’s reliability for evaluating trunk and LE alignment during a 45-degree SSC.Methods:25 healthy females (age 13.8 ± 1.4 years, mass 52.4 ± 9.3 kg, height 161.7 ± 6.0 cm) who regularly participated in cutting or pivoting sports were included. A repeated measures study design was used. Each subject performed three SSC trials with one trial randomly selected for analysis. Trials were recorded in the frontal and sagittal planes. Two physical therapist raters independently viewed and scored each video on two separate occasions. Videos orders were randomized and a two-week wash out period occurred between rounds. The E-CAST evaluates six movement variables (Figure 1). The assessment criteria for each variable was adapted to utilize the Dartfish motion analysis application on a smart phone (Figure 2). If a movement variable falls outside of the motion analysis definition, the subject scores a point, therefore a higher score represents poorer technique. Intraclass correlation coefficients (ICC) and 95% confident intervals (95% CI) were calculated for the E-CAST total score, and a kappa coefficient was calculated for each variable. Correlations were converted to z scores and compared for significance.Results:The cumulative intra-rater reliability was good (ICC= 0.821, 95% CI 0.687 – 0.898) and the cumulative inter-rater reliability was moderate (ICC= 0.752, 95% CI 0.565-0.859). Intra-rater kappa coefficients ranged from moderate to almost perfect for all variables (k= 0.505-0.875) and inter-rater kappa coefficients ranged from slight to good for all variables (k=0.336-0.733).Conclusion:The use of 2D kinematic analysis resulted in higher intra-and inter-rater reliability compared to the traditional qualitative E-CAST (ICC: 0.77, 95% CI 0.59-0.96 and ICC: 0.71, 95% CI 0.50-0.91), however, the difference was not significant (Zobs = -0.46 and Zobs = -0.30).

Reliability of the Expanded Cutting Alignment Scoring Tool (E-CAST) to Assess Trunk and Limb Alignment During a 45-Degree Side-Step Cut.

BackgroundCurrent clinical screening tools assessing risky movements during cutting maneuvers do not adequately address sagittal plane foot and ankle evaluations. The Cutting Alignment Scoring Tool (CAST) is reliable in evaluating frontal plane trunk and lower extremity alignment during a 45-degree side-step cut. The Expanded Cutting Alignment Scoring Tool (E-CAST) includes two new sagittal plane variables, knee flexion and ankle plantarflexion angle.Hypothesis/PurposeTo assess the inter-and intra-rater reliability of the E-CAST to evaluate trunk and lower extremity alignment during a 45-degree side-step cut.Study DesignRepeated MeasuresMethodsParticipants included 25 healthy females (13.8 ± 1.4 years) regularly participating in cutting or pivoting sports. Participants were recorded performing a side-step cut in frontal and sagittal planes. One trial was randomly selected for analysis. Two physical therapists independently scored each video using the E-CAST on two separate occasions, with randomization and a two-week wash-out between rounds. Observed movement variables were awarded a score of “1”, with higher scores representing poorer technique. Intraclass correlation coefficients (ICC) and 95% confident intervals (95% CI) were calculated for the total score, and a kappa coefficient (k) was calculated for each variable.ResultsThe cumulative intra-rater reliability was good (ICC=0.78, 95% CI 0.59-0.96) and the cumulative inter-rater reliability was moderate (ICC=0.71, 95% CI 0.50-0.91). Intra-rater kappa coefficients ranged from moderate to excellent for all variables (k= 0.50-0.84) and inter-rater kappa coefficients ranged from slight to excellent for all variables (k=0.20-0.90).ConclusionThe addition of two sagittal plane variables resulted in lower inter-rater ICC compared to the CAST (ICC= 0.81, 95% CI 0.64-0.91). The E-CAST is a reliable tool to evaluate trunk and LE alignment during a 45-degree side-step cut, with good intra-rater and moderate inter-rater reliability.Level of EvidenceLevel 2, Diagnosis

Translation of pain self-efficacy questionnaire into Gujarati language and analysis of psychometric properties in subjects with musculoskeletal pain

BACKGROUND: Musculoskeletal disorders (MSDs) are a leading cause of disability worldwide. Musculoskeletal conditions are typically characterized by pain which is often persistent and limits one's mobility, dexterity, and functional ability, reducing people's ability to work and participate in social roles with associated impacts on mental well-being. The Pain Self-Efficacy Questionnaire (PSEQ) is a 10-item questionnaire, developed to assess the confidence of people with ongoing pain; in performing activities with pain. The PSEQ scale is applicable to all persisting pain presentations. The study aims to translate PSEQ into Gujarati and analyze its psychometric properties in subjects with musculoskeletal pain.MATERIALS AND METHODS: Prior permission from the author was taken for the translation of PSEQ scale into Gujarati language. The translation procedure was carried out according to the guidelines provided by the WHO with forward and backward translation. Thereafter to analyze the psychometric properties of the scale, a total of 30 subjects (mean age of 45.43 ± 17.02) with different MSDs were enrolled in the study. The concurrent validity was obtained by establishing a correlation between the Gujarati translated version of PSEQ (PSEQ-G) and Visual Analog Scale (VAS). The intra-rater reliability was established by test-retest method within a span of 2 days. The questionnaire was again administered after 1 week by two different investigators to test the inter-rater reliability.RESULTS: The concurrent validity was established with a moderate strength negative correlation between PSEQ-G and VAS (r = -0.398, P < 0.05). The intra-rater reliability was found between 0.887 and 0.980 (intraclass correlation coefficient [ICC] = 0.955) and inter-rater reliability was between 0.848 and 0.958 (ICC = 0. 917). There was a high internal consistency with Cronbach's alpha 0.938. Bland-Altman plot analysis was performed for PSEQ-G scores between the two raters which showed agreement between the two scores within the limits of agreement with a 95% confidence interval. The linear regression showed no proportional bias.CONCLUSION: The results showed fair concurrent validity with a weak correlation between VAS and PSEQ-G scores (Gujarati version of PSEQ) and excellent inter-and intra-rater reliability for PSEQ-G in subjects with MSDs.

538 - Approach to Understanding the ‘Meaning’ of Behaviors in NCD: Beyond PIECES and GPA. Luthra’s Behavioral Assessment and Intervention Response Paradigm (LuBAIR™ Paradigm). Dr.

ABSTRACTBackground:With the incidence, prevalence, and cost of dementia care expected to rise, it has become crucial to develop a practical approach for managing behaviors in dementia. Presently non-pharmacological interventions, both interpersonal and environmental, are the gold standard for managing Behavioral and Psychological Symptoms of Dementia (BPSD). The purpose of the presentation is to reveal the reasons for paucity in developing effective pharmacological treatments for BPSD in moderate to advanced dementia and propose a new theoretical framework for labeling and classifying behaviors in moderate to advanced dementia. The LuBAIR paradigm will be less labor-intensive, more comprehensive, and improve the categorization of behaviors into clinically meaningful categories. It was also found that the LuBAIR Inventory has comparable inter-and intra-rater reliability and Construct and Criteria validity in comparison to BEHAV-AD and Cohen-Mansfield Agitation Inventory (CMAI).Methods:The literature on BPSD reviewed, focusing on terminology, models of behaviors, and identified deficiencies in both.Results:Terminology to describe moderate to advanced dementia behaviors lacks consistency, accuracy, and reliability in both research and clinical settings. Standardized scales currently utilized to diagnose clinical conditions also lack validity and reliability in moderate to advanced dementia. Models for understanding the occurrence of behaviors in dementia are dichotomized along the biological versus psychosocial paradigm. The reliability and validity of the LuBAIR Inventory were established in an earlier study and workshops, where it found that the LuBAIR was less labor-intensive, more comprehensive, and offered improved categorization of behaviors into clinically meaningful categories. It was also found that the LuBAIR Inventory has comparable inter-and intra-rater reliability and Construct and Criteria validity in comparison to BEHAV-AD and Cohen-Mansfield Agitation Inventory (CMAI).Conclusions:Deficiencies in existing terminology, assessment scales, and models are acknowledged. There are twelve newly formed behavioral categories to classify behaviors in moderate to advanced Dementia. These categories were used to develop a new behavioral assessment inventory titled LuBAIR (Luthra’s Behavioral Assessment and Intervention Response). The LuBAIR model will help clinical staff to understand the 'meaning’ of behaviors in persons with Dementia (PwD).

Validation of LensHooke® X1 PRO and Computer-Assisted Semen Analyzer Compared with Laboratory-Based Manual Semen Analysis

PurposeTo compare two automated semen quality analysis systems (LensHooke® X1 PRO [X1 PRO]) and IVOS CASA) for accuracy, precision and agreement with laboratory-based manual semen analysis (MSA).Materials and MethodsSemen samples (n=31) were obtained from normozoospermic healthy male volunteers and infertile men with a minimum abstinence period between 2–3 days. After complete liquefaction, 101 seminal aliquots were prepared and tested according to WHO 5th Edition (2010) guidelines. The results obtained by X1 PRO and IVOS CASA were compared with that of MSA. Additionally, 10 samples were used to evaluate the intra- and inter-rater agreement for X1 PRO and MSA.ResultsThe semen parameters (sperm concentration, total, and progressive motility) showed strong correlation and agreement for both automated semen analyzers and MSA (Spearman's rank correlation ≥0.92, p<0.0001). X1 PRO and IVOS CASA were able to differentiate samples with abnormal concentration with a positive predictive value (PPV) of 100%. Furthermore, the PPV for X1 PRO (86.5%) was higher than that for IVOS CASA (71.7%) in differentiating samples with abnormal motility. The X1 PRO device showed a high PPV (97.7%) in identifying normal sperm forms compared to MSA. Semen parameters evaluated showed a high inter-rater (kappa >0.91) and intra-rater (kappa >0.92) agreement for X1 PRO compared with MSA.ConclusionsBoth automated semen analyzers demonstrated a high level of concordance and their performance was comparable with MSA analysis. Furthermore, high-levels of inter-and intra-rater reliability for semen analysis indicate that the new X1 PRO can be used in a clinical laboratory to offer accurate and quick test results.

Correlation of Pulmonary Function to Novel Radiographic Parameters of Collapsing Parasol Deformity in Spinal Muscular Atrophy.

Spinal muscular atrophy (SMA) is a neuromuscular disease with manifestations of scoliosis, pulmonary function decline, and, uniquely, collapse of the ribs. Methods to quantify rib deformity and its impact on pulmonary function are sparse. The authors propose new radiographic measurements to quantify the aspect of SMA known as collapsing parasol deformity and correlate these measurements with pulmonary function. Twenty-eight full-spine radiographs of pediatric SMA patients were measured twice by 3 independent investigators, with 2 weeks separating each measurement. Radiographic measurements, demographics, spirometry results, and assisted ventilation rating were obtained. Twenty-one patients with spirometry metrics were assessed to correlate pulmonary function and spinal measurements. The intrarater intraclass correlation coefficient (ICC) for the measurements ranged from 0.706 to 0.99, and the interrater ICC ranged from 0.64 to 0.97. Eighteen of 19 variables had ICC values greater than 0.75 for inter- and intrarater reliability. Twenty-one patients with forced expiratory volume in 1 second and forced vital capacity were assessed in terms of these measurements. Ratio of the concave hemithoracic width at T6/convex hemithoracic width at T6 (P=.004) and ratio of convex vertical rib displacement at the apical rib/concave vertical rib displacement (P=.021) were both significantly correlated with decreased pulmonary function. No significant correlation was found examining the average vertical rib displacement at the apical rib. High inter-and intrarater reliability can be obtained in a variety of spinal measurements of SMA patients. Various measurements are correlated to diminished pulmonary function, specifically variables showing asymmetric changes in the chest cavity. [Orthopedics. 2021;44(2):e287-e293.].

Assessment of technical skill in hip fracture surgery using the postoperative radiograph

AimsTo develop a core outcome set of measurements from postoperative radiographs that can be used to assess technical skill in performing dynamic hip screw (DHS) and hemiarthroplasty, and to validate these against Van der Vleuten’s criteria for effective assessment.MethodsA Delphi exercise was undertaken at a regional major trauma centre to identify candidate measurement items. The feasibility of taking these measurements was tested by two of the authors (HKJ, GTRP). Validity and reliability were examined using the radiographs of operations performed by orthopaedic resident participants (n = 28) of a multicentre randomized controlled educational trial (ISRCTN20431944). Trainees were divided into novice and intermediate groups, defined as having performed < ten or ≥ ten cases each for DHS and hemiarthroplasty at baseline. The procedure-based assessment (PBA) global rating score was assumed as the gold standard assessment for the purposes of concurrent validity. Intra- and inter-rater reliability testing were performed on a random subset of 25 cases.ResultsIn total, 327 DHS and 248 hemiarthroplasty procedures were performed by 28 postgraduate year (PGY) 3 to 5 orthopaedic trainees during the 2014 to 2015 surgical training year at nine NHS hospitals in the West Midlands, UK. Overall, 109 PBAs were completed for DHS and 80 for hemiarthroplasty. Expert consensus identified four ‘final product analysis’ (FPA) radiological parameters of technical success for DHS: tip-apex distance (TAD); lag screw position in the femoral head; flushness of the plate against the lateral femoral cortex; and eight-cortex hold of the plate screws. Three parameters were identified for hemiarthroplasty: leg length discrepancy; femoral stem alignment; and femoral offset. Face validity, content validity, and feasibility were excellent. For all measurements, performance was better in the intermediate compared with the novice group, and this was statistically significant for TAD (p < 0.001) and femoral stem alignment (p = 0.023). Concurrent validity was poor when measured against global PBA score. This may be explained by the fact that they are measuring difference facets of competence. Intra-and inter-rater reliability were excellent for TAD, moderate for lag screw position (DHS), and moderate for leg length discrepancy (hemiarthroplasty). Use of a large multicentre dataset suggests good generalizability of the results to other settings. Assessment using FPA was time- and cost-effective compared with PBA.ConclusionFinal product analysis using post-implantation radiographs to measure technical skill in hip fracture surgery is feasible, valid, reliable, and cost-effective. It can complement traditional workplace-based assessment for measuring performance in the real-world operating room . It may have particular utility in competency-based training frameworks and for assessing skill transfer from the simulated to live operating theatre.Cite this article: Bone Joint Open 2020;1-9:594–604.

VALIDATION OF LensHookeTM X1 PRO AND COMPUTER-ASSISTED SEMEN ANALYZER COMPARED WITH LABORATORY-BASED MANUAL SEMEN ANALYSIS

To compare two automated semen quality analysis systems [LensHookeTM X1 PRO (X1 PRO) and IVOS CASA] for accuracy, precision and agreement with laboratory-based manual semen analysis (MSA). Semen analysis was conducted using X1 PRO (Bonraybio®), IVOS CASA (Hamilton Throne) and MSA according to WHO 5th Edition (2010) guidelines in andrology laboratory. Semen samples (n=31) were obtained from healthy male volunteers with normal semen parameters and infertile men with a minimum abstinence period of 2 to 3 days. After complete liquefaction, a total of 101 seminal aliquots were prepared and tested using X1 PRO, IVOS CASA and MSA. The test results obtained by X1 PRO and IVOS CASA were compared to MSA using Passing-Bablok regression analysis. Additionally, 10 samples were used to evaluate the intra- and inter-rater agreement for X1 PRO and MSA. The semen analysis results (sperm concentration, total motility, progressive (PR) motility, motile sperm concentration, and progressively motile sperm concentration) showed strong correlation and agreement for both automated semen analyzers (X1 PRO and IVOS CASA) and MSA with Pearson correlation above 0.92 (P<0.0001). X1 PRO and IVOS CASA were able to differentiate oligozoospermic samples with an accuracy of 100% and 93%, respectively. Furthermore, the positive predictive value for X1 PRO (86.5%) was higher than IVOS CASA (71.7%) in differentiating asthenozoospermic samples. Semen parameter (sperm concentration, total motility and PR motility) showed a high degree of inter- and intra-rater agreement evaluated using X1 PRO and MSA (Table 1). X1 PRO showed an intra-rater precision of coefficient variation, CV <15% for sperm concentration, total motility and PR motility, and was comparable with MSA. Both automated semen analyzers (X1 PRO and IVOS CASA) showed high level of accuracy and precision, and their performance was comparable with laboratory-based MSA. Furthermore, high-levels of inter-and intra-rater reliability for semen analysis indicates that new device, X1 PRO can be used in a clinical diagnostic laboratory to offer accurate test results.Tabled 1Table 1. Inter- and intra-rater agreement for semen parameters measured using LensHookeTM X1 PRO and MSASemen parametersInter-rater agreement*Intra-rater agreement**MSA (N=10)X1 PRO (N=10)MSA (N=10)X1 PRO (N=10)Concentration (106/mL)0.940.950.980.99Total motility (%)0.940.910.990.94Progressive motility (%)0.880.930.980.92*Three operators were involved. **Three readings were taken. Open table in a new tab

Ultrasound measurements of superficial and deep masticatory muscles in various postures: reliability and influencers

Masticatory muscle thickness provides objective measurements of the oral motor function, which may change in patients with oral myofascial pain. In this study, we aimed to establish a reliable ultrasound (US) protocol for imaging the superficial and deep masticatory muscles and to identify the potential influencers of the measurements. Forty-eight healthy participants without orofacial pain were enrolled. The intra-and inter-rater reliabilities of US measurements for masseter, temporalis, and lateral pterygoid muscles were assessed. Intraclass correlation coefficients for all muscles were greater than 0.6. The generalised estimating equation was used to analyse the impact of age, gender, laterality, and body mass index on the measurements, whereby age and body mass index were likely to be associated with an increase in masticatory muscle thickness. The thickness tended to be lesser in females. Laterality seemed to exert minimal influence on masticatory muscle thickness. Our study shows acceptable reliability of US in the evaluation of superficial and deep masticatory muscle thickness. Future studies are warranted to validate the usefulness of US imaging in patients with oral myofascial pain syndrome.

A Comparative Analysis of 150 Thumb Polydactyly Cases from the CoULD Registry Using the Wassel-Flatt, Rotterdam, and Chung Classifications

Three commonly used classifications for thumb polydactyly are the Wassel-Flatt, Rotterdam, and Chung. The ideal classification system would have high validity and reliability and be descriptive of the thumb anomaly. The purposes of this investigation were to (1) compare the inter- and intrarater reliability of these 3 classifications when applied to a large sample of patients enrolled in the Congenital Upper Limb Differences (CoULD) Registry and (2) determine the prevalence of radial polydactyly types when using the various classifications in a North American population. Inter- and intrarater reliability were determined using 150 cases of radial polydactyly presented in a Web-based format to 7 raters in 3 rounds, a preliminary training round and 2 observation rounds. Raters classified each case according to the Wassel-Flatt, Rotterdam, and Chung classifications. Inter- and intrarater reliability were evaluated with the intraclass correlation coefficient (ICC) calculated using 2-way random measures with perfect agreement. For Wassel-Flatt, both the interrater (ICC, 0.93) and the intrarater reliability (ICC, 0.91) were excellent. The Rotterdam classification had excellent reliability for both interrater reliability (ICC, 0.98) and intrarater reliability (ICC, 0.94), when considering type alone. Interrater analysis of the additional subtypes demonstrated a wide range of reliabilities. The Chung classification had good interrater (ICC, 0.88) and intrarater reliability (ICC, 0.77). Within the Wassel-Flatt classification, the most frequent unclassifiable thumb was a type IV hypoplastic thumb as classified by the Rotterdam classification. The Wassel-Flatt and Rotterdam classifications for radial polydactyly have excellent inter- and intrarater reliability. Despite its simplicity, the Chung classification was less reliable in comparison. The Chung and Rotterdam classification systems capture the hypoplastic subtypes that are unclassifiable in the Wassel-Flatt system. Addition of the hypoplastic subtype to the Wassel-Flatt classification (eg, Wassel-Flatt type IVh) would maintain the highest reliability and classify over 90% of thumbs deemed unclassifiable in the Wassel-Flatt system. The Wassel-Flatt and Rotterdam classifications have excellent inter-and intrarater reliability for the hand surgeon treating thumb polydactyly. Addition of a hypoplastic subtype to the Wassel-Flatt (Type 4h) allows classification of most previously unclassifiable thumbs.

Perth scoring system for assessment of the cleft lip

Background: Currently, there is no standardised assessment tool to assess facial aesthetics in cleft lip and palate surgery. Inter-centre comparison is hampered by the use of different aesthetic indices with low intra- and inter-rater reliability.&#x0D; Aim: The Perth scoring system is a new assessment tool for unilateral cleft lip which scores four key components of the cleft lip/nose repair: lip length, white roll, alar insertion point and vermillion. The aim of this study was to validate the Perth scoring system as a reliable and useful new assessment tool and to demonstrate the use of the scoring system to measure improvements after cleft lip revision.&#x0D; Method: Nineteen patients who underwent cleft lip revision by the senior author were selected. Pre- and postoperative photos were presented to a panel of raters to score. Scores were analysed to determine the intra-and inter-rater reliability and to compare outcomes.&#x0D; Results: Almost all patients (15/16) had improvement in scores (range 1.09–5.59) after cleft lip revision. Intra raters’ agreement scores from lowest to highest were: lip length (0.65), white roll (0.7), alar insertion point (0.78) and vermillion (0.78). The total intra-class correlation coefficient was 0.96 (0.94–0.98, 95% CI, P&lt;0.000).&#x0D; Conclusion: This new scoring system is a valid and useful tool for assessment of the unilateral cleft lip. The high rate of intra- and inter-rater reliability allow it to serve as a useful tool to compare surgical outcomes both within and between centres. Further field testing with a larger cohort of patients is required.

The Brazilian version of the Home Falls and Accidents Screening Tool (HOME FAST): translation, cross-cultural adaptation, validation and reliability

Abstract Objective: to translate and cross-culturally adapt the Home Falls and Accidents Screening Tool - HOME FAST into Brazilian Portuguese and to evaluate its construct validity and intra-and inter-rater reliability. Method: a cross-sectional study was carried out that included older people aged 60 years or older. Translation and cross-cultural adaptation were carried out in the following stages: 1. Translation, 2. Synthesis, 3. Back translation, 4. Expert panel (review and pre-final version), 5. Pre-testing, 6. Analysis by the expert panel and the final version of the instrument. The Berg Balance Scale - BBS was used to test construct validity (Spearman correlation coefficient). Additionally, intra-and inter-rater reliability analysis was conducted using the Intraclass Correlation Coefficient (ICC) and the Bland-Altman plot. Results were considered significant at p&lt;0.05. Results: the HOME FAST-Brazil was applied to 53 older people with a mean age of 71(5) years; 79% (42) of whom were female and 21% (11) of whom were male. The translation and cross-cultural adaptation process resulted in similar versions among translations. The correlation of the total score of HOME FAST-Brazil with the BBS was ρ=-0.241, p=0.041. The reliability rate was ICC=0.99 and 0.92 (intra-and inter-rater, respectively). Conclusion: The HOME FAST-Brazil, translated and cross-culturally adapted to Brazilian Portuguese, was shown to have construct validity and excellent intra-and inter-rater reliability.

Intra and inter-rater agreement of the Dynamic Movement Assessment ™ Agreement of the DMA™

Abstract Aims: To investigate the inter-and intra-rater agreement of the Dynamic Movement Assessment (DMA™) risk classification. Method: In this study, after the anthropometric measurements were made, 17 female soccer athletes were filmed performing the six DMA™ tests (full squat, step-up, single-leg squat, jump test, test plank, and side plank). Both, major and secondary deviations, were observed during the tests. Two experienced health professionals performed video analysis using Kinovea 8.15.0 (inter-rater agreement). To assess the intra-rater agreement, the same video analysis was performed two months later. Participants were rated from 0 to 21 points and at low, medium, moderate, and high risk of developing musculoskeletal injuries. To assess the reliability of the assessment of movement patterns of DMA, the intraclass correlation coefficient (ICC) was performed with a 2-way random-effects model with an absolute agreement (inter-rater) and a 2-way mixed-effects model and consistency (intra-rater). Weighted Kappa Agreement Analysis (kw) was performed with linear weights to assess the level of agreement related to the risk classification of DMA (high, moderate, medium, or minimum). The Analysis was performed with StatsDirect v.3 and SPSS (23.0). Results: Comparing the number of points between the inter-and intra-rater, the ICC was 0.91 (95% CI = 0.74-0.97) and 0.84 (95% CI = 0.59-0.94), respectively, with kw = 0.46 (P = 0.02) intra -rater and kw = 0.46 (P = 0.006) inter-rater (Table 9). Conclusion: DMA has excellent inter-and intra-rater reliability to evaluate movement patterns and classify the risk of musculoskeletal injuries.