Inter-rater Reliability Study Research Articles

Handheld dynamometry testing during dialysis: intra and inter-rater reliability study.

The aim was to assess the intra and inter-rater reliability of the handheld dynamometry testing of lower limb muscles during hemodialysis. This is a cross-sectional study including subjects undertaking hemodialysis for at least 3 months. Handheld dynamometer measurements of hip and ankle muscle strength (N) were registered on 4 different occasions, 2 trials by raters A and 2 by raters B, to evaluate the intra and inter-rater reliability. Bland-Altman plots were constructed to examine the agreement between raters by scoring the difference between raters against the mean of both raters' scores. The intraclass correlation coefficient (ICC) was used to assess relative intra-rater and inter-rater reliability. Absolute reliability was assessed with the minimal detectable change (MDC). Fifty-six participants (median age, 67 years) data showed high to very high relative reliability (ICCs 0.75 to 0.98). Regarding the intra-rater absolute reliability, MDC90 values ranged from 27.0 to 41.6.5 N (0.14 to 0.26 N/Kg m2) for hip muscles but increased for ankle muscles. Inter-rater values were wider. The mean difference between trials ranged narrowly across the two-time points for raters A (1.0 - 19.3 N) and for raters B (1.3 - 11.4 N). The mean difference between raters ranged from 2.4 to 13.5 N. The Bland-Altman plot revealed a systematic bias for raters A in ankle plantar flexion. . The assessment of the hip flexion and abduction strength during dialysis with a handheld dynamometer is reliable and safe. Normalized data provides more reliable results.

Interdisciplinary Consensus in Evaluating the Severity Subscale of the Original and Revised Toronto Western Spasmodic Torticollis Rating Scale Through Video-Based Assessment: An Inter-Rater Reliability Study.

Toronto Western Spasmodic Torticollis Rating Scale (TWSTRS) is widely employed for cervical dystonia (CD) evaluation. To assess the inter-rater reliability of the severity subscale of the original and revised TWSTRS using video recordings. Three raters, a PhD student with a nursing degree, a physiotherapist specialized in CD, and a neurologist-in-training independently rated all videos. The inter-rater reliability was assessed with the intra-class correlation coefficient (ICC). The total severity score of both tools demonstrated a good inter-rater reliability (ICC = 0.87 to 0.88). The inter-rater reliability of individual sub-items varied from poor (ICC = 0.29) to excellent (ICC = 0.9). The total severity score of both TWSTRS showed good inter-rater reliability in a multidisciplinary team, indicating their applicability for online patients' assessment. We recommend using the total subscale for outcome comparison. Furthermore, there is a need for more accurate definitions of duration factor and shoulder elevation.

Asymptotic Confidence Interval, Sample Size Formulas and Comparison Test for the Agreement Intra-Class Correlation Coefficient in Inter-Rater Reliability Studies.

The agreement intra-class correlation coefficient (ICCa) is a suitable statistical index for inter-rater reliability studies. With balanced Gaussian data, we prove the explicit form of ICCa asymptotic normality (ASN), valid both with analysis of variance (ANOVA), maximum likelihood (ML), or restricted ML (REML) estimates. An asymptotic confidence interval is then derived and its performances are examined by simulation compared to the most commonly used methods, under small, moderate and large sample size designs. Then, we deduce sample size calculation formulas, for the number of subjects and observers needed, to achieve a desired confidence interval width or an acceptable ICCa value test power and give concrete examples of their use. Finally, we propose a likelihood ratio test (LRT) to compare two ICCa's from two distinct subpopulations of patients (or raters) and study by simulation its first order risk and power properties. These methods are illustrated using data from two inter-rater reliability studies, one in physiotherapy with 42 patients and 10 raters and the second in neonatology with 80 subjects and 14 raters. In conclusion, we made recommendations to employ the proposed confidence interval for medium to large samples combined with the quantification of the minimal required sample size at the planning step, or the posterior-power at the analysis step, using simple dedicated formulas. Furthermore, with sufficient sizes, the proposed LRT seems suitable to compare inter-rater reliability between two patient subpopulations. Used wisely, this proposed methods toolbox can remedy common current issues in inter-rater reliability studies.

Intra- and Inter-Rater Reliability of Linear and Nonlinear Measures of Short-Term Heart Rate Variability Following Combat-Related Traumatic Injury.

Heart rate variability (HRV) is a marker of autonomic function. However, the reliability of short-term HRV measurement in individuals with combat-related traumatic injury (CRTI) remains undetermined. An intra- and inter-rater reliability study was conducted using a subsample (n = 35) of British servicemen with CRTI enrolled in the ongoing ADVANCE study. A five-minute epoch of single-lead electrocardiogram data collected during spontaneous breathing was used to measure HRV. HRV analyses were independently performed by two examiners using Kubios. Intraclass correlation coefficient (ICC), standard error of measurement (SEM), minimum detectable change (MDC), and coefficient of variance were calculated for linear [root mean square of successive difference (RMSSD), standard deviation of NN interval, low-frequency, high-frequency, total power] and nonlinear (SD1-2, acceleration and deceleration capacities, sample entropy) measures. Bland-Altman %plots were used to assess bias in intra- and inter-rater HRV data. The mean age of participants was 39.3 ± 6.3 years. An excellent ICC score of 0.9998 (95% CI 0.9997, 0.9999) was observed for intra-rater analyses of RMSSD, and similar excellent ICC scores were seen for all other HRV measures. The inter-rater reliability analyses produced an excellent ICC score (range 0.97-1.00). Comparatively, frequency-domain measures produced higher MDC% and SEM% scores than time-domain and nonlinear measures in both inter- and intra-rater analyses. The Bland-Altman plots revealed relatively higher bias for frequency-domain and nonlinear measures than time-domain measures. ECG-related short-term HRV measures were reliable in injured servicemen under spontaneous breathing. However, the reliability appeared better with the time-domain measure than frequency-domain and nonlinear measures in this sample.

Reliability testing of sonographic techniques in detecting lateral displacement of patellar tendon

ObjectivesPatellofemoral pain syndrome (PFPS), a common knee condition, is characterized as presence of peripatellar pain during activities especially during loading of the lower extremity. Gonzalez-Suarez et al. (2021) developed a sonographic technique that assessed patellar tendon displacement within the trochlear groove. This study aims to determine the reliability of this sonographic technique. MethodsIntra-rater and inter-rater reliability study of three sonologist on the lateral displacement of the patellar tendon using ultrasound. Patellar tendon displacement was assessed using the three novel sonographic measurement techniques: 1) region of trochlear groove where the patellar tendon midpoint is located, 2) distance between deepest portion of the trochlear groove and patellar tendon midpoint and 3) percentage of lateral displacement of the patellar tendon. The measurements were done with the patients in supine and standing. ResultsTen participants with a mean age of 27.6 ± 1.56 years were included in the study. Pilot studies were done to improve the sonographic technique. Intraclass correlation coefficients yielded good to excellent inter-rater and excellent intra-rater reliability (>0.75) in the supine and standing positions except the inter-rater reliability of the region of the trochlear groove where patellar tendon midpoint is in supine position. ConclusionThe sonographic techniques for measurement of lateral patellar tendon displacement showed good to excellent intra-rater and inter-rater reliability between sonologists of different levels of expertise using a well-defined scanning protocol.

How well can the fetal heart rate baseline be assessed by intrapartum intermittent auscultation? An interrater reliability and agreement study.

We aimed to examine the inter-reliability and agreement among midwives when assessing the fetal heart rate (FHR) using the handheld Doppler. The primary aim was to measure the reliability and agreement of FHR baseline (baseline) as beats per minute (bpm). The secondary aims were to measure fluctuations from the baseline, defined as increases and decreases, and classifications (normal or abnormal) of FHR soundtracks. This is the first interrater reliability and agreement study on intermittent auscultation (IA) to our knowledge. The participant population consisted of 154 women in labor, from a mixed-risk population and admitted to hospital for intrapartum care. The rater population were 16 midwives from various maternity care settings in Norway. A total of 154 soundtracks were recorded with a handheld Doppler device, and the 16 raters assessed 1-min soundtracks once, through an online survey (Nettskjema). They assessed the baseline, FHR increase or decrease, and the FHR classification. The primary outcome, baseline, was measured with intraclass correlation coefficient (ICC). The secondary outcomes were measured with kappa and proportion of agreement. The interrater reliability for the baseline (bpm) was ICC(A,1) 0.74 (95% CI 0.69-0.78). On average, an absolute difference of 7.9 bpm (95% CI 7.3-8.5 bpm) was observed between pairs of raters. Our results demonstrate an acceptable level of reliability and agreement in assessing the baseline using a handheld Doppler.

Reliability of Goniometric Techniques for Measuring Hip Flexor Length Using the Modified Thomas Test.

The modified Thomas test (MTT) is commonly used to assess the flexibility of hip musculature, including the iliopsoas, rectus femoris, and tensor fascia latae. This measurement is important to include in a comprehensive musculoskeletal examination. However, existing research shows conflicting results regarding its reliability, particularly due to variations in controlling pelvic tilt during testing, which may lead to inaccurate measurements of hip extension when quantifying the test outcomes. This study aimed to evaluate the intra- and inter-rater reliability of the Modified Thomas Test (MTT) in assessing hip flexor length using a goniometer. It was hypothesized that controlling for pelvic tilt would enhance the reliability of these measurements. Intra- and inter-rater reliability study. Sixty-four healthy individuals were recruited to participate in this study. The MTT was performed twice on each leg by both an experienced and a student physical therapist. Blinded goniometric measurements for hip extension range of motion (ROM) in the MTT position were taken with neutral pelvic tilt being enforced via palpation. A double-blind protocol was used where both examiners were unaware of each other's measurements and the goniometer was covered to blind the measuring therapist to the values as well. ROM values were entered into a Microsoft Excel spreadsheet and quantified using SPSS software. Statistical analysis included calculating Intraclass Correlation Coefficients (ICCs) and Standard Errors of Measurement (SEMs) using SPSS software. The study included 64 participants (mean age = 23.7 ± 4.34 years). The MTT demonstrated high intra-rater reliability (ICC = 0.911) and inter-rater reliability (ICC = 0.851). The SEMs indicated minimal variability around the mean scores. The average hip extension ROM measured was 5.43± 9.73 degrees. These results suggest that the MTT is a reliable tool for assessing hip flexor length in clinical practice, particularly when pelvic tilt is controlled. These results have important implications for accurately testing orthopedic limitations that can contribute to low back, hip, and knee pain. 3.

The Chinese Mandarin COMHON Index and Braden Scale to assess pressure injury risk in intensive care: An inter-rater reliability and convergent validity study

BackgroundThe COMHON Index is an intensive-care-specific pressure injury risk assessment tool, which has demonstrated promising psychometric properties. It has been translated into Chinese Mandarin but requires inter-rater reliability testing and comparison to the standard care instrument (Braden Scale) before clinical use. ObjectivesThis study aimed to test and compare the inter-rater reliability and convergent validity of the Chinese Mandarin versions of the COMHON Index and Braden Scale. MethodsThe study was conducted in a Chinese comprehensive intensive care unit. Based on a sample size calculation, five registered nurse raters with at least 6-months experience independently conducted risk assessments for 20 adult patients using both the COMHON Index and Braden Scale. Intraclass correlations (ICC) for inter-rater reliability, standard errors of measurement (SEM), and minimally detectable change (MDC) were calculated. Convergent validity was assessed using Pearson Product Moment Correlation for sum scores and Spearman's rho for subscales. ResultsInter-rater reliability of COMHON Index and Braden Scale sum scores was very high (ICC [1,1] = 0.973; [95% confidence interval 0.949–0.988]; SEM 0.54; MDC 1.50) and high (ICC [1,1] = 0.891; [95% confidence interval 0.793-0.951]; SEM 0.93; MDC 2.57), respectively. All COMHON-Index subscales demonstrated ICC values >0.6, whereas two Braden Scale subscales (Mobility, Activity) were below this threshold. Instrument sum scores were strongly correlated (Pearson's r = −0.76 [r2 = 0.58]; p < 0.001), as were three subscale item pairs (mobility rs= −0.56 [r2 = 0.32]; nutrition rs= −0.63 [r2 = 0.39]; level of consciousness/sensory perception rs= −0.67 [r2 = 0.45] p < 0.001). ConclusionBoth the COMHON Index and Braden Scale demonstrated high levels of inter-rater reliability and measured similar constructs. However, the COMHON Index demonstrated superior inter-rater reliability and the results suggest that it better detects changes in patient condition and subsequently pressure injury risk. Further testing is recommended.

A Multicenter Training and Interrater Reliability Study of the BASED Score for Infantile Epileptic Spasms Syndrome.

The best possible outcomes in infantile epileptic spasms syndrome require electroclinical remission; however, determining electrographic remission is not straightforward. Although the determination of hypsarrhythmia has inadequate interrater reliability (IRR), the Burden of AmplitudeS and Epileptiform Discharges (BASED) score has shown promise for the reliable interictal assessment of infantile epileptic spasms syndrome. Our aim was to develop a BASED training program and assess the IRR among learners. We hypothesized moderate or better IRR for the final BASED score and the presence or absence of epileptic encephalopathy (+/-EE). Using a web-based application, 31 learners assessed 12 unmarked EEGs (length 1-6 hours) from children with infantile epileptic spasms syndrome. For all readers, the IRR was good for the final BASED score (intraclass correlation coefficient 0.86) and +/-EE (Marginal Multirater Kappa 0.63). For all readers, the IRR was fair to good for all individual BASED score elements. These findings support the use of our training program to quickly learn the BASED scoring method. The BASED score may be a valuable clinical and research tool. Given that the IRR for the determination of epileptic encephalopathy is not perfect, clinical acumen remains paramount. Additional experience with the BASED scoring technique among learners and advances in collaborative EEG evaluation platforms may improve IRR.

Metrology of Supraspinatus Tendon Thickness in Swimmers with Tendinopathy Using Ultrasound Imaging: An Intra- and Inter-Rater Reliability Study.

Objective: This study aimed to assess the relative and absolute intra- and inter-rater reliability of supraspinatus tendon (SST) thickness. Materials: Thirty adolescent swimmers with supraspinatus (SS) tendinopathy (n = 15) and a control-matched group (n = 15) were evaluated. Tendon thickness was measured according to four different measure procedures, i.e., (1) at 15 mm, (2) at 10, 20, and 30 mm, (3) at 10, 15, and 20 mm, and (4) 5 and 10 mm lateral to the most hyperechogenic reference point of the biceps tendon. Each examiner took two US images for the test measurements with a 10 min rest period. After 30 min, the subjects underwent retest measurements that were also repeated 1 week later. Results: SST thickness was greater in swimmers with SS tendinopathy compared with the matched control group for each procedure and rater (p < 0.001). Intra- and inter-rater reliability was good to excellent (ICC2.3: 0.78-0.98 and 0.83-0.97, respectively) in both groups. The lowest intra- and inter-rater reliability was found in procedures no. 2 and 4 (ICC2.3: 0.78 and 0.83). However, procedure no. 3 was the most reliable with the lowest error rate (ICC2.3: 0.92-0.97; SEM: 0.05-0.10 mm; MDC: 0.14-0.28 mm). Conclusions: The study confirmed the diagnostic value of ultrasound in SS tendinopathy. A multiple-reference-point procedure including a simple methodology (10, 15, and 20 mm from biceps tendon), was defined as the most reliable, expressed by the highest intra- and inter-rater ICCs.

Interrater Reliability of Performing a Step-by-Step Procedure for Selected Pain Provocation Tests for Hamstrings and Special Tests for Other Lower Extremity Musculoskeletal Injuries

Objective: Our study aims to establish interrater reliability in performing the step-by-step procedure of selected pain provocation tests for hamstrings and special tests for lower extremity musculoskeletal injuries. Study Design : An interrater reliability study Setting: University of Santo Tomas - Sports Science Laboratory Participants: Ten healthy adults (five females, five males; age = 22.2 ± 0.42) from the university community. Main Outcome Measures: Interrater reliability of performing step-by-step procedures for selected pain provocation tests for hamstrings (painful resisted knee flexion 90°, painful resisted knee flexion 30°, active slump test, Puranen-Orava Test, bent knee stretch) and special tests for lower extremity musculoskeletal injuries (Lachman’s test, McMurray’s test, posterior drawer test, valgus, and varus stress test). Results: Fleiss kappa showed perfect agreement (κ = 1.00) for all test procedures except for Lachman’s test procedure 1 (κ= -0.11 [95% CI, -0.36 to 0.14]), active slump test procedure 4 (κ= -0.03 [95% CI, -0.28 to 0.23]), active slump test procedure 5 (κ= -0.11 [95% CI, -0.28 to 0.23]), and active slump test procedure 6 (κ= -0.05 [95% CI, -0.31 to 0.20]), which resulted in negative agreements. Conclusions: The researcher developed protocols for each special and provocative test were consistent in measuring the intended procedures, and the raters were generally consistent with their ability to measure these tests. Keywords: Interrater reliability, special tests, pain provocation tests, lower extremity injuries

Self-assessment using 30-second chair stand test for patients with knee osteoarthritis – an intra- and inter-rater reliability study

Background We aimed to evaluate the intra-rater reliability in 30-second chair stand test (30CST) as a self-test, and the inter-rater reliability comparing a self-test with a physiotherapist-assessed 30CST for patients with knee osteoarthritis (KOA). The second purpose was to evaluate the classification ability of the 30CST as self-test to distinguish between reduced physical function and normal function. Methods Patients with KOA performed two self-tests of 30CST at home and thereafter a physiotherapist assessed the patient. Absolute agreements were presented with intraclass correlation coefficient (ICC), 95% confidence interval (CI) and standard error of measurement (SEM). Post-hoc analyses were conducted to calculate minimal detectable change (MDC). The classification ability was analysed with receiver operating characteristic (ROC)-curves and area under the ROC-curve (AUC). The significance level was p < 0.05. Results In total, 114 followed the protocol. Intra-rater and inter-rater reliability were good (ICC 0.97, CI 0.95–0.99, SEM 0.89 and ICC 0.81, CI 0.72–0.87, SEM 2.29 respectively). MDC were 2.5 stands when comparing self-test results individually and 0.23 on a group level. The MDC between self-tests and physiotherapist assessment were 6.4 stands for individual comparison and 0.59 for group comparison. The self-test was 80% likely to detect reduced physical function compared to a physiotherapist assessing physical function with 30CST (AUC = 0.79–0.80). Conclusions The results indicate that 30CST can be a reliable self-assessment of physical function in lower extremities for patients with KOA. These findings can be useful in areas such as self-screening for reduced physical function, self-assessment in digital physiotherapy, and as measurement in research projects.

Ultrasound Examination of the Median Nerve at Different Levels: Inter-rater Reliability Study

Ultrasound Examination of the Median Nerve at Different Levels: Inter-rater Reliability Study

Reliability and validity of the medial standing overhead arm reach (SOAR) test as a measure of functional hip adduction motion

Objective The Posterior Standing Overhead Arm Reach (SOAR) test has been previously reported as a reliable clinical measure of closed chain hip extension motion. The proposed Medial SOAR test expands on that testing approach to provide a similar measure of functional hip adduction motion. This was a preliminary intrarater and interrater reliability and validity study of the Medial SOAR test as a measure of functional hip adduction. Design Cross-sectional. Setting University motion analysis laboratory. Participants Fifty hips were assessed in 25 (22 female) asymptomatic participants (mean age = 23.4 years, SD = 0.8). Main measures Maximum hip adduction during the Medial SOAR test was measured with a standard goniometer independently by two examiners. The test was also performed using three-dimensional motion capture. The intrarater and interrater reliability of the goniometric measure was determined using intraclass correlation coefficients, and the relationship between measures obtained via goniometry and three-dimensional motion capture was assessed with Pearson correlations and Bland–Altman analysis. Results Intrarater reliability (ICC2,3) was 0.88 (95% CI = 0.80–0.92) for Examiner 1 and 0.87 (95% CI = 0.79–0.92) for Examiner 2. The standard error of measurement and minimal detectable change were less than 3.0°. Interrater reliability demonstrated an intraclass correlation coefficient = 0.62 (95% CI = 0.28–0.79). Pearson correlations were significant with low-to-moderate associations (r = 0.49, P < 0.001; r = 0.24, P = 0.045). Conclusions Similar to the previously reported Posterior SOAR test, the Medial SOAR test demonstrated acceptable intrarater and interrater reliability, along with low-to-moderate associations with three-dimensional motion capture. The Medial SOAR test has the potential to provide a reliable and accurate assessment of closed chain hip adduction.

Nonaneurysmal perimesencephalic subarachnoid hemorrhage on noncontrast head CT: An accuracy, inter-rater, and intra-rater reliability study

Background and purposeTo evaluate the reliability and accuracy of nonaneurysmal perimesencephalic subarachnoid hemorrhage (NAPSAH) on Noncontrast Head CT (NCCT) between numerous raters. Materials and methods45 NCCT of adult patients with SAH who also had a catheter angiography (CA) were independently evaluated by 48 diverse raters; 45 raters performed a second assessment one month later. For each case, raters were asked: 1) whether they judged the bleeding pattern to be perimesencephalic; 2) whether there was blood anterior to brainstem; 3) complete filling of the anterior interhemispheric fissure (AIF); 4) extension to the lateral part of the sylvian fissure (LSF); 5) frank intraventricular hemorrhage; 6) whether in the hypothetical presence of a negative CT angiogram they would still recommend CA. An automatic NAPSAH diagnosis was also generated by combining responses to questions 2–5. Reliability was estimated using Gwet's AC1 (κG), and the relationship between the NCCT diagnosis of NAPSAH and the recommendation to perform CA using Cramer's V test. Multi-rater accuracy of NCCT in predicting negative CA was explored. ResultsInter-rater reliability for the presence of NAPSAH was moderate (κG = 0.58; 95%CI: 0.47, 0.69), but improved to substantial when automatically generated (κG = 0.70; 95%CI: 0.59, 0.81). The most reliable criteria were the absence of AIF filling (κG = 0.79) and extension to LSF (κG = 0.79). Mean intra-rater reliability was substantial (κG = 0.65). NAPSAH weakly correlated with CA decision (V = 0.50). Mean sensitivity and specificity were 58% (95%CI: 44%, 71%) and 83 % (95%CI: 72 %, 94%), respectively. ConclusionNAPSAH remains a diagnosis of exclusion. The NCCT diagnosis was moderately reliable and its impact on clinical decisions modest.

Interrater agreement and reliability of a palpation method for locating C1 transverse process in the cervical spine

ObjectivesTo examine the agreement and interrater reliability of a palpation method for locating the C1 transverse processes (C1TP) and the association between personal characteristics with the observed agreement. MethodsSingle-blinded, inter-rater reliability study. Ninety-nine participants (58 women, age 44 ± 10 years, body mass index 26.1 ± 4.3 kg/m2, neck disability index = 8.4 ± 7.3 points) were enrolled in this study. Two raters blinded to each other's assessment performed the palpation for locating the C1TP and marked it with invisible ultra-violet ink. A research assistant measured the distance between the skin marks labeled as C1TP. The association between sex, age, body mass index, neck pain intensity and disability on the interrater agreement was verified using a point-serial correlation coefficient. ResultsAbsolute and percent agreement for the right and left body sides were 90/99 (90.9 %, 95%CI 83.4 to 95.8) and 96/99 (97.0 %, 95%CI [91.4 to 99.4]), respectively. Interrater reliability was excellent for both right (Gwet's AC1 0.883, 95%CI [0.881 to 0.885]) and left body sides (Gwet's AC1 0.894, 95%CI [0.894 to 0.896]). After adjustment for multiple comparisons, no statistical evidence of correlation was observed for age, sex, body mass index, and NDI with the agreement for location of C1TP (r = −0.206 or lower, adjusted P-value = 0.328 or higher). ConclusionsThe palpatory method for the location of C1TP has excellent interrater agreement and reliability. Age, sex, body mass index, and disability show weak-to-negligible correlation with the agreement for location of C1TP. Implication for practice•A palpation method should be accurate and reliable for being introduced in clinical and laboratory settings•This is the first study to report interrater reliability results for locating C1 and the reliability is excellent•The result of our study increases our confidence in recommending the location of C1 in clinical practices when performing mobility tests for C0–C1–C2, the great nerve occipital palpation, as a start landmark for suboccipital muscle palpation and when applying direct manual procedures to restore mobility in C1–C2 segments as muscle energy technique, high-velocity low amplitude (HVLA) manipulation and mobilization with movement techniques•These results benefit the manual therapist's education process as palpatory skills and surface anatomy are core subjects in manual therapy education, including osteopathy, despite the lack of validation of most palpation methods

Measurement of Three-Dimensional Back Shape of Normal Adults Using a Novel Three-Dimensional Imaging Mobile Surface Topography System (MSTS): An Intra- and Inter-Rater Reliability Study

Postural and spinal deformities are major contributing factors to musculoskeletal (MSK) disorders. Posture screening and assessment can help to identify early morphological deformities, thereby preventing progression and reducing or correcting them with effective treatments. The study evaluates both intra- and inter-repeatability of using a mobile structured light sensor with a structured light pattern for building an accurate 3D human model and its use in postural screening. 16 young males (age: 25 ± 5.6 years, height: 172 ± 5.3 cm, mass: 69 ± 8.6 kg) participated without any musculoskeletal pain or pre-existing leg or spinal abnormalities. An iPad-based 3D mobile scanning tool, Structure SensorTM (2018 version), was used to capture the participants’ back and whole-body shape. The collected data (3D model) were realigned and processed in the open-source software, Netfabb BasicTM (7.2 version). For each participant, five trained raters individually measured three trials of standing back and body posture on two separate occasions to calculate both intra- and inter-rater reliability. With the use of this software, nine postural variables and angular displacements were individually measured by the raters. The results indicated good to excellent intra-rater and good to moderate inter-rater reliability for measuring 78% (7 out of 9) of postural variables with an ICC ranging from 0.70 to 0.98. The remaining 22% of variables (2 out of 9; lateral pelvic tilt and right frontal knee angle) showed moderate to low inter- and intra-rater reliability, with ICCs ranging from 0.26 to 0.79.

Comparison between observer-based and AI-based reading of CBCT datasets: An interrater-reliability study

ObjectiveTo assess the performance of human observers and convolutional neural networks (CNNs) in detecting periodontal lesions in cone beam computed tomography (CBCT), a total of 38 datasets were examined. Three human readers and a CNN-based solution were employed to evaluate the presence of periodontal pathologies in these datasets. Materials and MethodsDatasets were acquired with a Veraview X800 L P (JMorita Mfg. Corp., Kyoto, Japan). Three general dentists, previously calibrated by a general principal investigator, read the datasets in 3D MPR mode using Horos(LGPL license at Horosproject.org and sponsored by Nimble Co LLC d/b/a Purview in Annapolis, MD, USA) as a DICOM reader. All pathological changes including vertical bone loss, furcation involvement, and periradicular osteolysis were detected. Furthermore, the same datasets were analyzed automatically by Diagnocat (Diagnocat LLC, Prague, Czech Republic), a deep CNN. Finally, the performance of the dentists and the CNN were compared and evaluated. ResultsThe CNN’s performance was significantly lower compared to the human readers in the search for different types of lesions. The human observers achieved good to very good interobserver agreement, except for the evaluation of the vertical lesions, which resulted in a moderate agreement. ConclusionThe CNN used in this study was found to be ineffective in identifying periodontal lesions and was not adequately trained to offer significant assistance in the automated evaluation of periodontal lesions in CBCT datasets.

Assessment of the bearing-down manoeuvre in pregnancy and detection of paradoxical levator ani muscle contraction using 2D transperineal ultrasound and vaginal palpation: a concurrent validity and inter-rater reliability study.

To examine the concurrent validity and inter-rater reliability of vaginal palpation as a measure of the quality of the bearing-down manoeuvre (BDM) and the detection of a paradoxical levator ani muscle contraction (LAM) in pregnant women, compared with 2D transperineal ultrasound (TPUS). Concurrent validity and inter-rater reliability study. Physiotherapy clinic. Twenty pregnant women in their third trimester. The anterior posterior diameter (APD) was measured during the BDM using TPUS by one experienced physiotherapist. An APD that shortened by >2 mm from rest was described as LAM shortening, an APD that moved by 0-2 mm was described as no change and an APD that lengthened by >2 mm was described as LAM lengthening. Vaginal palpation described the LAM during the BDM as no movement, shortening or lengthening. Participants were allowed two attempts and the best attempt was measured. APD using TPUS and the assessor's subjective description of LAM during the BDM using vaginal palpation. TPUS detected more paradoxical LAM contractions during the BDM than palpation. Agreement between vaginal palpation and TPUS assessment for BDM was poor. The Fleiss kappa coefficients were 0.457 (90% CI 0.16-0.71) between TPUS and one assessor and 0.326 (90% CI 0.01-0.6) between TPUS and the other assessor. In addition, inter-rater reliability was poor between observers palpating the BDM, with a Fleiss kappa coefficient of 0.375 (90% CI 0.13-0.64). This study did not find vaginal palpation of the BDM in pregnant women to have concurrent validity or inter-rater reliability. Clinicians should be aware of potential inaccuracies when palpating the BDM, and, where possible, seek an assessment via TPUS.

Assessment of frailty by paramedics using the clinical frailty scale - an inter-rater reliability and accuracy study

BackgroundFrailty assessment by paramedics in the prehospital setting is understudied. The goals of this study were to assess the inter-rater reliability and accuracy of frailty assessment by paramedics using the Clinical Frailty Scale (CFS).MethodsThis was a cross-sectional study with paramedics exposed to 30 clinical vignettes created from real-life situations. There was no teaching intervention prior to the study and paramedics were only provided with the French version of the CFS (definitions and pictograms). The primary outcome was the inter-rater reliability of the assessment. The secondary outcome was the accuracy, compared with the expert-based assessment. Reliability was determined by calculating an intraclass correlation coefficient (ICC). Accuracy was assessed through a mixed effects logistic regression model. A sensitivity analysis was carried out by considering that an assessment was still accurate if the score differed from no more than 1 level.ResultsA total of 56 paramedics completed the assessment. The overall assessment was found to have good inter-rater reliability (ICC = 0.87 [95%CI 0.81–0.93]). The overall accuracy was moderate at 60.6% (95%CI 54.9–66.1) when considering the full scale. It was however much higher (94.8% [95%CI 92.0–96.7] when close assessments were considered as accurate. The only factor associated with accurate assessment was field experience.ConclusionThe assessment of frailty by paramedics was reliable in this vignette-based study. However, the accuracy deserved to be improved. Future research should focus on the clinical impact of these results and on the association of prehospital frailty assessment with patient outcomes.RegistrationThis study was registered on the Open Science Framework registries (https://doi.org/10.17605/OSF.IO/VDUZY).