Good Inter-rater Reliability Research Articles

Development and validation of an observational tool to measure engagement in telepractice early intervention

Purpose To develop and validate an evidence-based observational tool to measure engagement in telepractice early intervention (EI). Materials and Methods The Engagement in Telepractice Observational Tool (eTOT) was developed based on a previous e-Delphi study. Two independent raters used a set of 50 items to score 45 videorecorded telepractice EI sessions with young children with communication difficulties and their family. Principal component analysis (PCA) and reliability measures were conducted. Results PCA identified six key components related to the measurement of engagement in telepractice. The components representing professional behaviours were (1) interacting directly with the parent; (2) getting to know the child and family to keep them motivated; (3) maximising the telepractice environment; and (4) demonstrating and practising the parent’s role as agent of therapy. The two remaining components represented family behaviours: (5) parent interacting and supporting their child; and (6) family collaborating directly with the professional. The eTOT was found to have good reliability (α = .86), with very good test-retest and inter-rater reliability scores (κw = .85 and .94 respectively). Conclusion There is preliminary evidence that the eTOT is a valid and reliable tool to measure engagement during telepractice EI. The eTOT can be used as a self-reflection tool to identify and provide engagement-building opportunities in family-centred telepractice EI.

Intra-rater and inter-rater reliability of a handheld myotonometer measuring myofascial stiffness of lower lumbar myofascial tissue in healthy adults.

Biomechanical muscle properties, such as stiffness, can be valuable indicators of tissue health and show promise as a diagnostic and treatment measure for chronic low back pain (CLBP). The development of accessible assessment technologies, such as the MyotonPRO portable device, allows for the convenient quantification of muscle tone and stiffness changes. The aim of this study is to assess the reliability of lumbar erector spinae muscle stiffness with the MyotonPRO in healthy adults and to compare stiffness changes between prone and sitting position. Thirty asymptomatic participants (N = 15 women and N = 15 men) aged between 18 and 65 years were recruited to participate in this study. Two examiners tested muscle stiffness at the palpable muscle belly, one finger away from the spinous process at the level of the L4 vertebra, first from the left and then from the right side, both in prone position and after in sitting position. For inter-rater reliability, all participants were tested by two examiners on the same day, and intra-rater reliability was calculated using the same examiner's assessment results with an exact 24-h interval. Intraclass correlation coefficients (ICC), standard error measures (SEM) and minimum detectable change (MDC) with a 95% confidence interval were calculated to assess intra-rater and inter-rater reliability. Statistical analysis revealed good intra-rater reliability with an ICC of 0.88 (95% CI [0.76-0.94]) for the stiffness of the left erector spinae and excellent intra-rater reliability with an ICC of 0.91 (95% CI [0.82-0.95]) for the right erector spinae, both in the prone position. Intra-rater reliability in the sitting position was excellent to very good with an ICC of 0.91 (95% CI [0.82-0.96]) for the left side and an ICC of 0.89 (95% CI [0.78-0.95]) for the right side. The results for the left-sided prone position showed good inter-rater reliability with an ICC of 0.87 (95% CI [0.73-0.94]). The prone position on the right side also showed good inter-rater reliability with an ICC of 0.84 (95% CI [0.68-0.92]). The inter-rater reliability for the left and right side in the sitting position was excellent with an ICC of 0.96 (95% CI [0.92-0.98]) for the left side and an ICC of 0.95 (95% CI [0.90-0.97]) for the right side. This study demonstrated high reliability in measuring lumbar erector spinae muscle stiffness with the MyotonPRO in healthy adults and the ability of the device to detect even small changes in erector spinae muscle stiffness, testing both the right and left sides and measuring in both prone and sitting positions. The use of the sitting position to assess lumbar tissue tension in individuals may serve as a valuable substitute for the prone position, particularly for patients who experience discomfort in the prone position, and could have additional practical significance in clinical settings.

The role of mental health nursing in pediatric hematology/oncology - Part 3: Evaluating feasibility, acceptability, and appreciation.

In February 2022, we introduced the innovative new role of mental health nurse clinician (MHNC) in pediatric hematology/ oncology in a pediatric hospital in Quebec with the aim of better addressing the needs of young cancer patients and their families. In two previous articles, we explained the method used to develop the role, as well as the strategy employed to implement it. In this article, we seek to evaluate the effectiveness of the implementation strategy by measuring role feasibility. We will also examine role acceptability and appreciation 1 year following implementation. We asked three groups of participants (adolescents, parents and care providers) to complete a research questionnaire based on the Theoretical Framework of Acceptability (TFA), with good inter-rater reliability and discriminant validity. The questionnaire, which was made available online, consisted of statements with Likert-scale responses (from 1 to 5), as well as open questions. We performed a standard descriptive analysis and used the responses to the open questions to support or add nuance to the quantitative results. In total, 31 healthcare professionals, 10 adolescent cancer patients and 10 parents of adolescent cancer patients filled out the questionnaire. Between February 2022 and February 2023, the MHNC provided a total of 691 interventions to 112 individuals (patients and parents). Follow-up sessions were offered to 61 individuals, and 99.6% of them accepted. We found a very strong correlation (r = 0.96) between the number of hours worked by the MHNC and the number of interventions provided. The vast majority of participants from all three groups agreed that the role is appropriate, appreciated and necessary, and that it brings much-needed clinical expertise in mental health to the department. Care providers, adolescents, and parents indicated that they would request this service again and would recommend it to others, if it were needed. The results, therefore, demonstrate excellent feasibility and high levels of acceptability and appreciation. Our study shows that the implemented MHNC role helps improve the clinical management of the mental health of adolescents with cancer, as well as their parents. The role brings much-needed expertise in psychiatric nursing care and supports the clinical practice of care providers. Another study will be conducted in the near future to measure the effectiveness of the MHNC's interventions.

Interdisciplinary Consensus in Evaluating the Severity Subscale of the Original and Revised Toronto Western Spasmodic Torticollis Rating Scale Through Video-Based Assessment: An Inter-Rater Reliability Study.

Toronto Western Spasmodic Torticollis Rating Scale (TWSTRS) is widely employed for cervical dystonia (CD) evaluation. To assess the inter-rater reliability of the severity subscale of the original and revised TWSTRS using video recordings. Three raters, a PhD student with a nursing degree, a physiotherapist specialized in CD, and a neurologist-in-training independently rated all videos. The inter-rater reliability was assessed with the intra-class correlation coefficient (ICC). The total severity score of both tools demonstrated a good inter-rater reliability (ICC = 0.87 to 0.88). The inter-rater reliability of individual sub-items varied from poor (ICC = 0.29) to excellent (ICC = 0.9). The total severity score of both TWSTRS showed good inter-rater reliability in a multidisciplinary team, indicating their applicability for online patients' assessment. We recommend using the total subscale for outcome comparison. Furthermore, there is a need for more accurate definitions of duration factor and shoulder elevation.

Ultrasound Speckle Tracking Method Based on Gradient Optical Flow to Quantify Small Longitudinal Displacement, Shear and Longitudinal Strain in Peripheral Nerves

ObjectiveThis study aimed to develop, validate and test the clinical feasibility of ultrasound (US) speckle tracking method based on gradient optical flow for quantifying small longitudinal displacements, shear and strain in peripheral nerves. MethodsThe speckle tracking method was validated using seven thawed, fresh-frozen isolated cadaveric forearms. Longitudinal motion of the median nerve was captured using a high-frequency 22 MHz linear probe. An air bubble marker was inserted as a reference point for manual measurement comparison. The precision and accuracy of the method were assessed by comparing manual and automatic measurements. Clinical feasibility was tested on eight healthy subjects, measuring the longitudinal displacement of the median nerve during elbow extension and shoulder anteflexion. ResultsThe method demonstrated linearity, high precision and accuracy, particularly with a backtrace of five frames, reducing the displacement underestimation to 4%. In cadaveric models, the highest shear strain was observed at the nerve-tissue interfaces. In healthy subjects, the mean displacement of the median nerve was 3.3 ± 1.0 mm, with good inter-rater reliability (intraclass correlation coefficient = 0.87). ConclusionThe US speckle tracking method based on gradient optical flow effectively quantifies small longitudinal displacements and shear strain in peripheral nerves, with high precision and accuracy. However, the method could not detect longitudinal strain in nerves within the range of tested displacements. Future studies should investigate its applicability to smaller and deeper nerves and its usefulness in different pathological conditions.

Reliability and usability of a novel inertial sensor-based system to test craniocervical flexion movement control

BackgroundNeck pain has a significant global impact, ranking as the fourth leading cause of disability. Recurrent neck pain often leads to impaired sensorimotor control, particularly in craniocervical flexion (CFF). The Craniocervical Flexion Test (CCFT) has been widely investigated to assess the performance of deep cervical flexor muscles. However, its use requires skilled assessors who need to subjectively detect compensations, progressive increases in range of motion (ROM) or excessive superficial flexor activation during the test. The aim of this study was to design and develop a novel Craniocervical Flexion Movement Control Test (CFMCT) based on inertial sensor technology and real-time computer feedback and to evaluate its safety and usability, as well as inter and intra-rater reliability in both healthy individuals and patients with neck pain.MethodsWe used inertial sensor technology associated with new software that provides real-time computer feedback to assess CCF movement control through two independent test protocols, the progressive consecutive stages protocol (including progressive incremental stages of ROM) and the random stages protocol (providing dynamic and less predictable movement patterns). We determined intra and inter-rater reliability and standard error of the measurement for both protocols. The participants rated their usability was analysed through the System Usability Scale (SUS) and possible secondary effects associated with the tests were registered.ResultsThe progressive consecutive stages protocol and the random stages protocol were safe and easy to use (SUS scores of 82.00 ± 11.55 in the pain group and 79.56 ± 13.36 in the asymptomatic group). The progressive consecutive stages protocol demonstrated good inter-rater reliability (intraclass correlation coefficient [ICC] ≥ 0.75) and moderate to good intra-rater reliability (ICC 0.62–0.80). However, the random stages protocol exhibited lower intra-rater reliability, especially in the neck pain group, where the reliability values were poor in some cases (ICC 0.48–0.72).ConclusionThe CFMCT (progressive consecutive stages protocol) is a promising instrument to evaluate CCF motor control in patients with chronic neck pain. It has potential for telehealth assessment and easy adherence for exercise prescription and seems to be a safe and usable tool for patients with pain and asymptomatic individuals. Its use as a test or for exercise needs to be further investigated to facilitate its transfer to clinical practice.

Validation of the Japanese version of Algoplus® for the assessment of acute postoperative pain in older patients with cancer.

This study aimed to validate a Japanese version of the Algoplus® tool by assessing postoperative pain in older Japanese patients with cancer and examining the scale's psychometric properties. After translating Algoplus® into Japanese, we conducted a cross-sectional study of patients aged 65 years or older who underwent surgery to remove malignant tumors. Two registered nurse-certified investigators used the Numerical Rating Scale, the Japanese version of Algoplus®, and the Japanese version of the Abbey Pain Scale before and after analgesic use on postoperative days 3 and 5 to evaluate response to pharmacologic therapy. Validity was tested by a correlation analysis between the Japanese version of Algoplus®, two pain scales, and nine hypotheses related to demographic variables and surgical invasions. The Kuder-Richardson-20 test and Cohen's Kappa coefficient were used for internal consistency and inter-rater reliability, respectively. The total score of the Japanese version of Algoplus® showed strong to moderate correlations with the two pain scales. Demographic characteristics were not associated with the total score of the Japanese version, but significant correlations with operative time and postoperative analgesia administration existed. The scale demonstrated good internal consistency (Kuder-Richardson-20 α: .70) and inter-rater reliability (Kappa coefficient .72). The total score of the Japanese version decreased significantly after analgesic use in both postoperative assessments. The Japanese version of Algoplus® is a reliable and valid instrument for nurses to easily assess acute postoperative pain in older Japanese patients with cancer and shows good responsiveness for detecting the change in pain status.

Intrarater and Interrater Reliability of Quantitative Ultrasound Speed of Sound by Trained Raters at the Distal Radius in Postmenopausal Women.

Quantitative ultrasound (QUS) performed at the distal radius shows potential as an effective tool for evaluating fracture risk, especially in postmenopausal women, but the reliability of this tool has not been reported in this patient population. This study investigated the within-day intrarater and interrater reliability of trained raters performing QUS measures at the distal radius of postmenopausal community-dwelling women. Eighteen women between the ages of 56 and 87years were recruited from senior centers in the Greenville area of South Carolina and consented to participate. A QUS bone sonometer was used in this study; the trained raters performed 3 repeated speed of sound (SOS) measures at the nondominant radius of each participant on the same day. Intraclass correlation coefficients (ICCs) were calculated (model 3,1 for intrarater and 2,3 for interrater reliability) to determine the reliability within and between raters. In addition, we calculated the standard error of measurement (SEM) and the minimal detectable change at 95% confidence interval (MDC 95 ). The within-day intrarater reliability was excellent for both physical therapist raters (ICC (3,1) ranging from 0.93 to 0.91). The SEM and MDC for the within-day intrarater reliability measures were small for both raters (SEM=36.79-38.75 or 0.92%-0.97% of the pooled mean; MDC 95 =101.99-107.40). The interrater reliability for SOS measurements using radial QUS was good with an ICC (2,3) of 0.77 (95% CI, 0.36-0.92) for 3 averaged measures. The SEM and MDC were larger for the interrater reliability measures (SEM=62.79 or 1.57% of the pooled mean; MDC 95 =174.05). Quantitative ultrasound SOS measurement at the distal radius showed excellent within-day intrarater and good interrater (when 3 measures were averaged) reliability in community-dwelling postmenopausal women. The SEM and MDC 95 were also low for interrater reliability when a mean of 3 trials was used. This study is the first to demonstrate that this particular QUS device at the radius may be a reliable fracture risk screening tool for postmenopausal women.

Accuracy and reliability of the Ipswich touch test in identifying loss of protective sensation among diabetic patients

ObjectiveTo evaluate the accuracy of the Ipswich touch test compared to the 10-g monofilament test for identifying of loss of protective sensation in Thai patients with diabetes mellitus. MethodsA cross-sectional observational study was conducted on Thai patients with diabetes mellitus who attended routine annual foot check-ups in an outpatient diabetes clinic. The loss of protective sensation was assessed by the Ipswich touch test and compared with the Semmes-Weinstein 10-g monofilament test. Sensitivity, specificity, predictive values and likelihood ratios were calculated to measure the accuracy of the Ipswich touch test against 10-g monofilament as a reference standard. The interrater reliability of the Ipswich touch test was assessed by two raters. ResultsIn a study of 283 diabetic patients, 10-g monofilament detected a 25 % prevalence of loss protective sensation. The Ipswich touch test demonstrated a good diagnostic accuracy, with a sensitivity of 70.8 %, specificity of 98.6 %, and an area under the curve of 0.85 when compared to 10-g monofilament. Positive and negative predictive values were 94.4 % and 90.8 % respectively. The positive likelihood ratio was 49.82, and the negative likelihood ratio was 0.30. Interrater reliability, assessed with two raters in a subset of 93 participants, yielded a kappa of 0.88, indicating almost perfect agreement. ConclusionsThe Ipswich Touch Test demonstrated good accuracy and interrater reliability compared to the standard 10-g monofilament, thus establishing its effectiveness as a valuable diagnostic tool for identifying loss of protective sensation among diabetic patients. However, its relatively low sensitivity suggests it should be used with caution as a screening tool.

Sexual Orientation Microaggression Rating Scale (SOMRS): Development and association with alliance ruptures.

Sexual minority clients report experiencing frequent microaggressions during therapy, however, therapists may not recognize those microaggressions or may be reluctant to self-report them. The main aim of the present study was thus to develop an observational measure of in-session therapist-committed microaggressions related to the sexual orientation of sexual minority individuals (e.g., those who identify as lesbian, gay, bisexual, or queer). The present study further examined the association between therapist-committed sexual orientation microaggressions and ruptures in the therapeutic alliance. We hypothesized that clinically significant microaggressions would be positively associated with withdrawal ruptures in the alliance. The sample consisted of 44 gay and bisexual men who participated in a cognitive behavioral treatment designed to reduce depression, anxiety, human immunodeficiency virus-transmission-risk behaviors, and substance use. An observer-based coding measure designed for this study, the Sexual Orientation Microaggression Rating Scale (SOMRS), was utilized to capture sexual minority microaggressions in the initial sessions of treatment. Good interrater reliability was achieved for the SOMRS. Microaggressions were coded in 34% of the sessions. Within the subset of sessions with coded microaggressions, a significant association was found between withdrawal ruptures and microaggression significance ratings. The SOMRS holds potential for supporting research on microaggression as well as future efforts to help clinicians recognize and repair in-session behaviors that negatively impact sexual minority clients. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Reliability and validity of the Chinese version of the Structured Interview of Personality Organization-Revised (STIPO-R).

With the publication of the fifth edition of the Diagnostic and Statistical Manual of Mental Disorders, a set of dimensional criteria was added as an emerging alternative model to the diagnosis of personality disorder (PD; American Psychiatric Association, 2013). Parallel to this, within the object relations conceptualization of personality pathology, a structured interview, the Structured Interview of Personality Organization (STIPO), was developed to assess pathological personality and then revised (STIPO-R). In this study, the reliability and validity of the Chinese version of the STIPO-R were tested on a sample of 236 Chinese participants, including both psychiatric patients and healthy individuals. Overall, the STIPO-R showed good internal consistency, interrater and test-retest reliability, and generally satisfactory results in structure and convergent validity. The STIPO-R also demonstrated discriminant validity (healthy individuals vs. psychiatric patients with PD vs. psychiatric patients without PD). Results are also discussed in light of cultural differences between Chinese and Western cultures. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Validity of an expert-based job exposure matrix of hand-wrist physical exposures and their prospective associations with carpal tunnel syndrome.

Epidemiological studies of occupational risk factors for rare disorders require large study populations with adequate exposure estimates. Job exposure matrices (JEMs) linked to national information on standardized job titles may enable such large studies. We aimed to establish and validate a JEM for occupational hand-wrist exposures that could be linked to Danish national register data on job titles and hand-wrist disorders. We developed a JEM for hand-wrist repetition, force, vibration, and computer work in 96 job groups covering 91% of the 2227 occupational titles in the Danish version of the International Standard Classification of Occupation-88, and examined inter-rater reliability of five expert ratings. Poisson regression models were used to estimate incidence rate ratios for the association of carpal tunnel syndrome (CTS) with the level of repetitive movements, force, vibration, and hours of computer work described by the JEM, adjusted for relevant confounders. The JEM based on expert ratings had fair to good interrater reliability. The incidence of CTS increased with increasing levels of force, hand-wrist repetition, and vibration, Exposure-response patterns for repetition and vibration became less consistent after adjustment for force. The interaction between repetition and force was complex and did not support an overall positive interaction. Computer work was negatively associated with incident CTS. The JEM was able to identify known risk factors for CTS consistent with current evidence, and provided further associations on exposure-response patterns, mutual exposure adjustment, and interaction effects between repetition and force. The reliability of expert assessments of hand-wrist physical exposures was fair to good.

Diagnosing Sarcopenia with AI-Aided Ultrasound (DINOSAUR)-A Pilot Study.

Background: Sarcopenia has been recognized as a determining factor in surgical outcomes and is associated with an increased risk of postoperative complications and readmission. Diagnosis is currently based on clinical guidelines, which includes assessment of skeletal muscle mass but not quality. Ultrasound has been proposed as a useful point-of-care diagnostic tool to assess muscle quality, but no validated cut-offs for sarcopenia have been reported. Using novel automated artificial intelligence (AI) software to interpret ultrasound images may assist in mitigating the operator-dependent nature of the modality. Our study aims to evaluate the fidelity of AI-aided ultrasound as a reliable and reproducible modality to assess muscle quality and diagnose sarcopenia in surgical patients. Methods: Thirty-six adult participants from an outpatient clinic were recruited for this prospective cohort study. Sarcopenia was diagnosed according to Asian Working Group for Sarcopenia (AWGS) 2019 guidelines. Ultrasonography of the rectus femoris muscle was performed, and images were analyzed by an AI software (MuscleSound® (Version 5.69.0)) to derive muscle parameters including intramuscular adipose tissue (IMAT) as a proxy of muscle quality. A receiver operative characteristic (ROC) curve was used to assess the predictive capability of IMAT and its derivatives, with area under the curve (AUC) as a measure of overall diagnostic accuracy. To evaluate consistency between ultrasound users of different experience, intra- and inter-rater reliability of muscle ultrasound parameters was analyzed in a separate cohort using intraclass correlation coefficients (ICC) and Bland-Altman plots. Results: The median age was 69.5 years (range: 26-87), and the prevalence of sarcopenia in the cohort was 30.6%. The ROC curve plotted with IMAT index (IMAT% divided by muscle area) yielded an AUC of 0.727 (95% CI: 0.551-0.904). An optimal cut-off point of 4.827%/cm2 for IMAT index was determined with a Youden's Index of 0.498. We also demonstrated that IMAT index has excellent intra-rater reliability (ICC = 0.938, CI: 0.905-0.961) and good inter-rater reliability (ICC = 0.776, CI: 0.627-0.866). In Bland-Altman plots, the limits of agreement were from -1.489 to 1.566 and -2.107 to 4.562, respectively. Discussion: IMAT index obtained via ultrasound has the potential to act as a point-of-care evaluation for sarcopenia screening and diagnosis, with good intra- and inter-rater reliability. The proposed IMAT index cut-off maximizes sensitivity for case finding, supporting its use as an easily implementable point-of-care test in the community for sarcopenia screening. Further research incorporating other ultrasound parameters of muscle quality may provide the basis for a more robust diagnostic tool to help predict surgical risk and outcomes.

Lung ultrasound to predict the duration of respiratory support in newborn infants with respiratory distress.

Lung ultrasound (LU) and clinical parameters evaluated during the first postnatal hour potentially predict the length of CPAP therapy in newborns with respiratory distress. In a single-centre, prospective observational pilot study, 130 newborns ≥36 weeks gestational age were assessed using standardised LU at 30 and 60 min postnatally. Various clinical parameters were evaluated influencing CPAP duration (<1 vs. ≥1 h) using univariate and multivariate analyses. Lung ultrasound score >5, FiO2 > 0.21 and respiratory acidosis 30 min postnatally were associated with CPAP ≥1 h. Our model showed good diagnostic quality (ROC AUC = 0.87) and was confirmed by classification and regression tree (CART) analysis. Additional LU findings like double lung point and pleural line abnormalities were frequently observed, with good interrater reliability for LU interpretation (ICC = 0.76-0.77). Newborns with postnatal respiratory distress at risk for prolonged CPAP therapy can be identified based on specific LU and clinical parameters assessed 30 min postnatally. Despite the need for validation in an independent sample, these findings may lay the groundwork for a prediction tool. LU proved feasible and reliable for assessing respiratory status in this population, highlighting potential utility in clinical practice.

Field Hip Stability Isometric Test (F-HipSIT): Reliability of Assessing the Hip Posterolateral Muscle Strength in Sports Settings

Context: The Hip Stability Isometric Test (HipSIT) is commonly employed in clinical settings for evaluating the hip posterolateral muscle strength. In this study, we introduced the “Field Hip Stability Isometric Test” (F-HipSIT) and assessed the intrarater and interrater reliability of this strength assessment specifically designed for sports settings. Design: Reliability study. Methods: Two independent raters (A and B) went to athletes’ training facilities to conduct 2 sessions of F-HipSIT spaced at least 1 week apart. The average peak force value from 3 valid attempts of each leg was recorded and normalized by the participant’s body mass for statistical analysis. Results: Thirty male and 30 female amateur athletes took part in this study. Rater A obtained similar values in the first (0.39 [0.05] and 0.44 [0.07] kg·f/kg) and second (0.39 [0.06] and 0.45 [0.07] kg·f/kg) testing days for men and women, respectively. Rater B also found similar values in the first (0.35 [0.06] and 0.42 [0.08] kg·f/kg) and second (0.36 [0.06] and 0.45 [0.08] kg·f/kg) testing days for men and women, respectively. Excellent intrarater intraclass correlation coefficient (ICC) values were found for men (ICC = .922) and women (ICC = .930), with coefficient of variation of 6% to 8% and minimal detectable change of 0.06 to 0.10 kg·f/kg. The F-HipSIT presented good interrater reliability for men (ICC = .857) and women (ICC = .868), with coefficient of variation of 5% and minimal detectable change of 0.05 to 0.06 kg·f/kg. Conclusion: The F-HipSIT intrarater and interrater reliability among male and female recreational athletes supports this field test as a quick and convenient screening tool to monitor hip posterolateral muscle strength in sports settings.

A Reliability Study of the Load Distribution Percentage While Walking Using Curalgia Feet Sx Smart Insoles.

Background Gait analysis has evolved through many years of research. Many methods are used to analyze the gait of a subject. Recent times have showna high demand for wearable sensor-based insoles integrated with smartphone-based devices used for gait analysis due to ease of use. This study utilized Curalgia Feet Sx Smart Insoles and its software toolset, Gait Analysis+, designed and manufactured in India making it an accessible and cost-effective option. The Curalgia Feet Sx Smart Insoles allow for a broad range of biofeedback-based rehabilitation and recovery training for several patients and have many applications, such as sports performance enhancement and neurological disorder rehab (e.g., brain stroke rehab). The system also significantly delays the onset of neurodegenerative illnesses by providing balance and proprioceptive training. The smart insole can help the athlete, the coach, and the sports medicine team get the on-field data in real-time, which will help them understand if any technical or biomechanical alterations are required. This may help in performance enhancement. This study aimed to determine the interrater reliability of the load distribution percentage parameter of the Curalgia Feet Sx Smart Insole for both feet while walking in a controlled setting. Methodology A total of 120 subjects were enrolled in the study. In total, 90 subjects were randomly selected using Research Randomizer which included male and female students and staff at Sardar Bhagwan Singh University. The subjects were asked to come to the research lab of the physiotherapy department wearing their sports shoes. Curalgia Feet Sx insoles were inserted into the shoe firmly to fit properly. Two assessors took two readings after the smart insole was connected to the smartphone-based application, GaitAnalysis+, via Bluetooth. The dynamic analysis option was selected, and each subject's analysis was done one after another with a desirable break in between. Each subject walked for three minutes at their normal speed after pressing "Start Analysis." At the three-minute mark, the subjects were asked to press "Stop Analysis" and the investigator downloaded the report on the smartphone. The data collected was compiled as the cumulative weight in kg (load distribution) borne and the % weight (load distribution %) borneby each foot for the duration of the walk. Statistical analysis was done using Karl Pearson's test and interclass correlation calculation. Results Assessor 1 and Assessor 2 collected readings for the left foot as "L" and the right foot as "R." Assessor 1 readings were L1-R1 for load distribution and L1% and R1% for load distribution %. Assessor 2 readings were L2-R2 for load distribution and L2% and R2% for load distribution %. The r value (correlation coefficient) was calculated using the load distribution. The mean value of L1 was 337.46 (SD=94.16). The mean L2 was 313.6 (SD=104.40). The R1 mean was 229.03 (SD=112.88), and the R2 mean was 233.011 (SD=79.84). The r was 0.7171 for the left foot and 0.7502 for the right foot, suggesting an excellent correlation. The ICC was calculated for load distribution %. The means of L1% was 55.94, L2% was 57.59, R1% was 44.06, and R2% was 42.41. The ICC was found to be 0.91 for both feet, suggesting high interrater reliability for the tested parameter. Conclusions The findings confirmed that the Curalgia Feet Sx Smart Insoles presented good interrater reliability for the load distribution % parameter.

Development and Psychometric Examination of a New Social Competence Outcome Measure for Children with Autism Spectrum Disorder: The Observational Social Competence Assessment.

Current assessments of social competence for children with autism spectrum disorder (ASD) are mostly designed for screening or diagnosis, not for measuring outcomes. This study aimed to develop a professional-administrated outcome measure, the Observational Social Competence Assessment (OSCA), and examine its psychometric properties. The OSCA was constructed based on a multidimensional view of social competence (i.e., social skill elements, social reciprocity, and social adjustment). For psychometric evaluation, 89 children with ASD between 3 and 12 years (mean = 70.69 months, SD = 15.31) were assessed with the OSCA and with assessments of ASD symptoms, verbal comprehension ability, and adaptive function. The results show that the OSCA has good internal consistency (Cronbach's α = 0.820-0.954), test-retest reliability (intraclass correlation coefficients [ICC] = 0.917-0.960), and inter-rater reliability (ICC = 0.905-0.974). The OSCA also has good convergent (r = 0.508-0.703, p < 0.01) and divergent validity (r = 0.105, p = 0.496), as well as good responsiveness to changes in the social adjustment dimension (Cohen's d = 1.26 and standardized response mean [SRM] = 1.92). Conclusively, these results show that the OSCA is sufficiently reliable, valid and responsive to be applied as an outcome measure of social competence in children with ASD.

Design and Interrater Reliability of the Pediatric Version of the Race Scale: PedRACE.

Acute ischemic stroke is a leading cause of pediatric death and disability. A clinical scale adapted for children can ensure early detection of candidates for urgent acute ischemic stroke treatment. The Rapid Arterial Occlusion Evaluation (RACE) scale for adults, which scores 5 items (facial palsy 0-2; arm motor function 0-2; leg motor function 0-2; head/gaze deviation 0-1; and aphasia or agnosia 0-2), has good sensitivity and specificity in detecting large vessel occlusion. We adapted the previously validated RACE scale for use in children as the Pediatric RACE scale. This adapted scale was tested by prehospital/emergency room staff attending to patients covered by the Catalan Pediatric Stroke Code and child neurologists for its correlation with the Pediatric National Institutes of Health Stroke Scale and for interrater reliability. The study included 50 children, 18 with confirmed strokes (7 acute ischemic strokes and 11 hemorrhagic strokes). Prehospital/emergency staff and child neurologists agreed fully regarding 82% of patients and 100% regarding head/gaze deviation and agnosia. The Pediatric RACE scale correlated strongly with the Pediatric National Institutes of Health Stroke Scale in evaluations by child neurologists (Spearman ρ, 0.852; P<0.001) and prehospital/emergency staff (Spearman ρ, 0.781; P<0.001). The median Pediatric RACE score was significantly higher in patients with large vessel occlusion (6.5; interquartile range, 6-7) than with other etiologies. Pediatric RACE, showing good interrater reliability and correlation with the Pediatric National Institutes of Health Stroke Scale, is a simple scale to detect candidates for pediatric acute stroke treatment, designed for both prehospital and in-hospital use by non-neurologist medical staff.

Reliability of the Test of Gross Motor Development Third Edition Among Children with Developmental Coordination Disorder

Aim The Test of Gross Motor Development Third Edition (TGMD-3) is used to assess the development of fundamental movement skills in children from 3 to 10 years old. This study aimed to evaluate the intra-rater, inter-rater, and test-retest reliability and to determine the minimal detectable change (MDC) value of the TGMD-3 in children with developmental coordination disorder (DCD). Methods The TGMD-3 was administered to 20 children with DCD. The child’s fundamental movement skills were recorded using a digital video camera. Reliability was assessed at two occasions by three raters using the generalizability theory. Results The TGMD-3 demonstrates good inter-rater reliability for the locomotor skills subscale, the ball skills subscale, and the total score (φ = 0.77 − 0.91), while the intra-rater reliability was even higher (φ = 0.94 − 0.97). Test-retest reliability was also shown to be good (φ = 0.79-0.93). The MDC95 was determined to be 10 points. Conclusion This study provides evidence that the TGMD-3 is a reliable test when used to evaluate fundamental movement skills in children with DCD and suggests that an increase of 10 points represents a significant change in the motor function of a child with DCD.

PECGreview: Assessment of a Novel Tool to Evaluate the Accuracy of Pediatric ECG Interpretation Skills.

The skill of interpretation of the electrocardiogram (ECG) remains poor despite existing educational initiatives. We sought to evaluate the validity of using a subjective scoring system to assess the accuracy of ECG interpretations submitted by pediatric cardiology fellows, trainees, and faculty to the Pediatric ECG Review (pECGreview), a web-based ECG interpretation training program. We conducted a retrospective, cross-sectional study of responses submitted to pECGreview. ECG interpretations were assessed independently by four individuals with a range of experience. Accuracy was assessed using a 3-point scale: 100% for generally correct interpretations, 50% for over- or underdiagnosis of minor ECG abnormalities, and 0% for over- or underdiagnosis of major ECG abnormalities. Inter-rater agreement was assessed using expanded Bland-Altman plots, Pearson correlation coefficients, and Intraclass Correlation Coefficients (ICC). 1460 ECG interpretations by 192 participants were analyzed. 107 participants interpreted at least five ECGs. The mean accuracy score was 76.6 ± 13.7%. Participants were correct in 66.1 ± 5.1%, had minor over- or underdiagnosis in 21.5 ± 4.6% and major over- or underdiagnosis in 12.3 ± 3.9% of interpretations. Validation of agreement between evaluators demonstrated limits of agreement of 11.3%. Inter-rater agreement exhibited consistent patterns (all correlations ≥ 0.75). Absolute agreement was 0.74 (95% CI 0.69-0.80), and average measures agreement was 0.92 (95% CI 0.89-0.94). Accuracy score analysis of as few as five ECG interpretations submitted to pECGreview yielded good inter-rater reliability for assessing and ranking ECG interpretation skills in pediatric cardiology fellows in training.