High Inter-rater Reliability Research Articles

Management of the Difficult Airway: An Appraisal of Clinical Practice Guidelines.

Management of the difficult airway can be a challenging process, which necessitates actionable recommendations from well-established guidelines. Herein, clinical practice guideline (CPG) quality is evaluated using the Appraisal of Guidelines for Research and Evaluation (AGREE II) instrument. A systematic literature search was performed using Scopus, EMBASE, and MEDLINE via PubMed. Literature database. Data were abstracted from relevant guidelines and appraised by 4 expert reviewers in the 6 domains of quality defined by AGREE II. Intraclass correlation coefficients (ICC) were calculated across domains to quantify interrater reliability. Twelve guidelines met the inclusion criteria. With a mean quality score of 83.1%, the highest quality guideline was authored by the American Society of Anesthesiologists (ASA). Low-quality content was observed in CPGs authored by the Japanese Society of Anesthesiologists (JSA) and the Chinese Collaboration Group for Emergency Airway Management (CCGEAM). Overall, deficits were most pronounced in domains describing the involvement of stakeholders, developmental rigor, and editorial independence. These findings were consistent among the panel of independent reviewers, with high ICC inter-rater reliability scores of 58.0% to 70.0% for the referenced domains. By providing a comprehensive appraisal of guidelines, this report may serve as a reference for clinicians seeking to understand and improve upon the developmental quality of difficult airway management resources. According to AGREE II criteria for the quality of the guideline creation process, the 2022 ASA guideline outperforms its predecessors.

Automated Patient Note Grading: Examining Scoring Reliability and Feasibility.

Scoring postencounter patient notes (PNs) yields significant insights into student performance, but the resource intensity of scoring limits its use. Recent advances in natural language processing (NLP) and machine learning allow application of automated short answer grading (ASAG) for this task. This retrospective study evaluated psychometric characteristics and reliability of an ASAG system for PNs and factors contributing to implementation, including feasibility and case-specific phrase annotation required to tune the system for a new case. PNs from standardized patient (SP) cases within a graduation competency exam were used to train the ASAG system, applying a feed-forward neural networks algorithm for scoring. Using faculty phrase-level annotation, 10 PNs per case were required to tune the ASAG system. After tuning, ASAG item-level ratings for 20 notes were compared across ASAG-faculty (4 cases, 80 pairings) and ASAG-nonfaculty (2 cases, 40 pairings). Psychometric characteristics were examined using item analysis and Cronbach's alpha. Inter-rater reliability (IRR) was examined using kappa. ASAG scores demonstrated sufficient variability in differentiating learner PN performance and high IRR between machine and human ratings. Across all items the ASAG-faculty scoring mean kappa was .83 (SE ± .02). The ASAG-nonfaculty pairings kappa was .83 (SE ± .02). The ASAG scoring demonstrated high item discrimination. Internal consistency reliability values at the case level ranged from a Cronbach's alpha of .65 to .77. Faculty time cost to train and supervise nonfaculty raters for 4 cases was approximately $1,856. Faculty cost to tune the ASAG system was approximately $928. NLP-based automated scoring of PNs demonstrated a high degree of reliability and psychometric confidence for use as learner feedback. The small number of phrase-level annotations required to tune the system to a new case enhances feasibility. ASAG-enabled PN scoring has broad implications for improving feedback in case-based learning contexts in medical education.

Interrater reliability of the Standing and Walking Assessment Tool for spinal cord injury.

Psychometric study. The Standing and Walking Assessment Tool (SWAT) is a standardized approach to the evaluation of standing and walking capacity following traumatic spinal cord injury (tSCI) in Canada. The SWAT classifies individuals with a tSCI into 12 stages of standing and walking capacity that are paired with well-established outcome measures, such as the Berg Balance Scale and 10-m Walk Test. Prior research has demonstrated the validity and responsiveness of the SWAT stages; however, the reliability remains unknown. The objective of this study was to evaluate the interrater reliability of the SWAT stages. Inpatient units of two Canadian rehabilitation hospitals. Adults with sub-acute tSCI were recruited. SWAT stage was evaluated for each participant by two physical therapists separately. The two therapists aimed to complete the evaluations within one day of each other. To evaluate interrater reliability, the percentage agreement between the SWAT stages rated by the two physical therapists was calculated, along with a linear weighted kappa statistic with a 95% CI. Forty-five individuals with sub-acute tSCI (36 males, 9 females, mean (SD) age of 54.8 (17.9) years) participated. The percentage agreement in SWAT stages between the two physical therapists was 75.6%. A kappa statistic of 0.93 with a 95% CI, 0.81-1.05 was obtained. In cases where therapists disagreed (18% of participants), therapists differed by 1-2 stages only. The SWAT stages have high interrater reliability, providing further support for the use of the SWAT in rehabilitation practice in Canada.

Qualitative visual assessment of the J-sign demonstrates high inter-rater reliability

ObjectivesTo assess the inter- and intra-rater reliability of the classification of the J-sign as “large” versus “small or none” as compared to another two-level system (“present” versus “absent”) and a three-level system (“large,” “small,” or “none”) and to identify anatomical and patient factors associated with the presence of a large J-sign. MethodsForty patients (40 knees) with recurrent patellar instability were prospectively enrolled and recorded on video actively extending their knee while seating. Four raters classified patellar tracking on two separate occasions using three systems: 1) two groups: J-sign versus no J-sign; 2) three groups: large J-sign, small J-sign, or no J-sign; and 3) two groups: large J-sign versus small or no J-sign. The intra- and inter-rater reliability of each system was assessed using kappa statistics. Anatomical (trochlear dysplasia, tibial tubercle-trochlear groove (TT-TG) distance, patellar height) and patient (Beighton score) factors as well as Knee Injury and Osteoarthritis Outcome Score (KOOS) subscales were compared between patients with a large J-sign and patients with a small or no J-sign. ResultsInter- and intra-rater reliability were found to be highest with the two-level classification system of a large J-sign versus a small or no J-sign (inter-rater kappa ​= ​0.76, intra-rater kappa ​= ​0.75). Patients with a large J-sign had more severe trochlear dysplasia as assessed with the sulcus angle (p ​= ​0.042) and were more likely to have a tight lateral retinaculum (p ​= ​0.032) and an elevated Beighton score (p ​= ​0.009). No significant differences in KOOS subscales were noted based on the presence of a large J-sign versus a small J-sign or no J-sign. ConclusionQualitative visual assessment of patellar tracking with the J-sign demonstrates substantial inter- and intra-rater reliability, particularly when utilizing a two-group classification system to identify knees with a large J-sign. Patients with a large J-sign demonstrate an increased incidence of a tight lateral retinaculum, generalized ligamentous laxity, and trochlear dysplasia. Level of evidenceLevel III - cross-sectional study.

An examination of psychometric properties of study quality assessment scales in meta-analysis: Rasch measurement model applied to the firefighter cancer literature.

Most existing quality scales have been developed with minimal attention to accepted standards of psychometric properties. Even for those that have been used widely in medical research, limited evidence exists supporting their psychometric properties. The focus of our current study is to address this gap by evaluating the psychometrics properties of two existing quality scales that are frequently used in cancer observational research: (1) Item Bank on Risk of Bias and Precision of Observational Studies developed by the Research Triangle Institute (RTI) International and (2) Newcastle-Ottawa Quality Assessment Scale (NOQAS). We used the Rasch measurement model to evaluate the psychometric properties of two quality scales based on the ratings of 49 studies that examine firefighters' cancer incidence and mortality. Our study found that RTI and NOQAS have an acceptable item reliability. Two raters were consistent in their assessment, demonstrating high interrater reliability. We also found that NOQAS has more items that show better fit than the RTI scale. The NOQAS produced lower study quality scores with a smaller variation, suggesting that NOQAS items are much easier to rate. Our findings accord with a previous study, which conclude that the RTI scale was harder to apply and thus produces more heterogenous quality scores than NOQAS. Although both RTI and NOQAS showed high item reliability, NOQAS items are better fit to the underlying construct, showing higher validity of internal structure and stronger psychometric properties. The current study adds to our understanding of the psychometric properties of NOQAS and RTI scales for future meta-analyses of observational studies, particularly in the firefighter cancer literature.

Engaging Adolescent Heart Transplant Recipients Through In-App Messaging During Mobile Health Intervention.

Introduction: Adolescents heart transplant recipients experience difficulty with adherence to immunosuppressive medication leading to increased risk of organ rejection, hospitalization, and mortality. Few interventions have been successful to promote medication adherence in adolescent heart transplant patients as most fail to engage the patient in the behavioral change process and support patient-provider communication. The purpose of this study was to explore the nature and degree of in-app communication between adolescent heart transplant recipients and nursing staff during an asynchronous mobile video directly observed therapy intervention. Methods: A content analysis of 894 in-app messages exchanged between 10 adolescent patients and nurses during a 12-week mobile-based intervention was conducted. Two authors used an inductive, iterative process to guide a thematic analysis of the asynchronous in-app messages with high interrater reliability ranging from 81.5% to 100%. Findings: In-app messages fell under 3 broad content categories: (1) interpersonal support and rapport-building, (2) medically related questions and information, or (3) functional information about the intervention and the mHealth app. Results demonstrated the degree to which interpersonal engagement occurred during the intervention, the nature of these exchanges, and their relationship to medication adherence. Conclusions: The study provided insights into the feasibility and benefits of 2-way communication features of the directly observed therapy intervention in promoting engagement and in improving medication adherence among adolescent heart transplant patients. Continued research and clinical focus on patient engagement and impactful aspects of interpersonal communication could aid in the translation of this intervention into standard clinical care at pediatric transplant centers.

Global assessment of surgical skills (GASS): validation of a new instrument to measure global technical safety in surgical procedures.

Broad implementation of the American Board of Surgery's entrustable professional activities initiative will require assessment instruments that are reliable and easy to use. Existing assessment instruments of general laparoscopic surgical skills have limited reliability, efficiency, and validity across the spectrum of formative (low-stakes) and summative (high-stakes) assessments. A novel six-item global assessment of surgical skills (GASS) instrument was developed and evaluated with a focus upon safe versus unsafe surgical practice scoring rubric. The GASS was developed by iterative engagement with expert laparoscopic surgeons and includes six items (economy of motion, tissue handling, appreciating operative anatomy, bimanual dexterity, achievement of hemostasis, overall performance) with a uniform three-point scoring rubric ("poor-unsafe", "adequate-safe", "good-safe"). To test inter-rater reliability, a cross-sectional study of four bariatric surgeons with experience ranging from 4 to 28years applied the GASS and the global operative assessment of laparoscopic skills (GOALS) to 30 consecutive Roux-en-Y gastric bypass procedure operative videos. Inter-rater reliability was assessed for a simplified dichotomous "safe" versus "unsafe" scoring rubric using Gwet's AC2. The GASS inter-rater reliability was very high across all six domains (0.88-1.00). The GASS performed comparably to the GOALS inter-rater reliability scores (0.96-1.00). The economy of motion and bimanual dexterity items had the highest percentage of unsafe ratings (9.2% and 5.8%, respectively). The GASS, a novel six-item instrument of general laparoscopic surgical skills, was designed with a simple scoring rubric (poor-safe, adequate-safe, good-safe) to minimize rater burden and focus feedback to trainees and promotion evaluations on safe surgical performance. Initial evaluation of the GASS is promising, demonstrating high inter-rater reliability. Future research will seek to assess the GASS against a broader spectrum of laparoscopic procedures.

Auditory-Perceptual and Pupillometric Evaluation of Vocal Roughness and Listening Effort in Tracheoesophageal Speech

This study evaluated auditory-perceptual judgments of perceived vocal roughness (VR) and listening effort (LE) along with pupillometric responses in response to speech samples produced by tracheoesophageal (TE) talkers. Twenty normal-hearing, naive young adults (eight menand twelve women) served as listeners. Listeners were divided into two groups: (1) a with-anchor (WA) group (four menand six women) and (2) a no-anchor (NA) group (four menand six women). All were presented with speech samples produced by twenty TE talkers; listeners evaluated two auditory-perceptual dimensions-VR and LE-using visual analog scales. Anchors were provided to the WA group as an external referent for their ratings. In addition, during the auditory-perceptual task, each listener's pupil reactions also were recorded with peak pupil dilation (PPD) measures extracted as a physiologic indicator associated with the listening task. High interrater reliability was obtained for both the WA and NA groups. High correlations also were observed between auditory-perceptual ratings of roughness and LE, and between PPD values and ratings of both dimensions for the WA group. The inclusion of an anchor during the auditory-perceptual task improved interrater reliability ratings, but it also imposed an increased demand on listeners. Data obtained offer insights into the relationship between subjective indices of voice quality (ie, auditory-perceptual evaluation) and physiologic responses (PPD) to the abnormal voice quality that characterizes TE talkers. Furthermore, these data provide information on the inclusion/exclusion of audio anchors and potential increases in listener demand in response to abnormal voice quality.

Convergent validity and inter-rater reliability of a lower-limb multimodal physical function assessment in community-dwelling older adults.

Introduction: Lower-limb physical function declines with age and contributes to a greater difficulty in performing activities of daily living. Existing assessments of lower-limb function assess one dimension of movement in isolation or are not time-efficient, which discourages their use in community and clinical settings. We aimed to address these limitations by assessing the inter-rater reliability and convergent validity of a new multimodal functional lower-limb assessment (FLA). Methods: FLA consists of five major functional movement tasks (rising from a chair, walking gait, stair ascending/descending, obstacle avoidance, and descending to a chair) performed consecutively. A total of 48 community-dwelling older adults (32 female participants; age: 71 ± 6years) completed the FLA as well as timed up-and-go, 30-s sit-to-stand, and 6-min walk tests. Results: Slower FLA time was correlated with a slower timed up-and-go test (ρ = 0.70), less sit-to-stand repetitions (ρ = -0.65), and a shorter distance in the 6-min walk test (ρ = -0.69; all, p < 0.001). Assessments by two raters were not different (12.28 ± 3.86s versus 12.29 ± 3.83s, p = 0.98; inter-rater reliability ρ = 0.993, p < 0.001) and were statistically equivalent (via equivalence testing). Multiple regression and relative weights analyses demonstrated that FLA times were most predicted by the timed up-and-go performance [adjusted R 2 = 0.75; p < 0.001; raw weight 0.42 (95% CI: 0.27, 0.53)]. Discussion: Our findings document the high inter-rater reliability and moderate-strong convergent validity of the FLA. These findings warrant further investigation into the predictive validity of the FLA for its use as an assessment of lower-limb physical function among community-dwelling older adults.

Intra-rater and inter-rater reliability of cranio cervical flexion test in symptomatic subjects of neck pain using pressure biofeedback

Objective: To investigate the intra – rater and inter – rater reliability of cranio cervical flexion test in symptomatic subjects. Background of study: Neck flexor muscles endurance was negatively correlated with cervical pain and dysfunction. Cranio cervical flexion test protocol has been advocated to train cranio cervical flexor muscle performance at present there is no consensus to the most effective method. The CCFT is a clinical test of neuromotor control of deep flexor of the cervical spine. Methodology: This was an observational study design with test-retest type and conducted in the ACS medical college and hospital. 50 samples were selected based on inclusion criteria. The CCFT was measured by one tester twice for intra rater reliability and two testers for inter-rater reliability were again tested after 24 Hours (Next day) by keeping time and environment optimal. In this CCFT, the activation pressure score and highest-pressure score was measured using Pressure Biofeedback unit. Result: The mean score for Activation Pressure by intra-rater tester one was found to be 2.24 (1.69) and 2.64 (1.48) in retest. The ICC intrarater score for Highest pressure value by tester one was 23.92 (2.17) and 24.32 (1.99) in retest. The mean score for activation pressure by Inter – rater examiners was found to be 2.92(1.15) by examiner - 1 and 2.72(1.19) by examiner - 2. The ICC Inter – rater score for highest pressure value by examiner - 1 was 24.32(1.95) and 24.4(2.33) by examiner - 2. The ICC value of the study indicated high reliability of the activation pressure and highest-pressure value among the intra and inter-raters. Conclusion: The CCFT has high inter-rater and intra-rater reliabilities in symptomatic subjects. Bangladesh Journal of Medical Science Vol. 22 No. 03 July’23 Page : 617-622

Reliability and Agreement of Automated Head Measurements From 3-Dimensional Photogrammetry in Young Children.

This study aimed to assess the reliability and agreement of automated head measurements using 3-dimensional (3D) photogrammetry in young children. Specifically, the study evaluated the agreement between manual and automated occipitofrontal circumference (OFC) measurements (n = 264) obtained from 3D images of 188 patients diagnosed with sagittal synostosis using a novel automated method proposed in this study. In addition, the study aimed to determine the interrater and intrarater reliability of the automatically extracted OFC, cephalic index, and volume. The results of the study showed that the automated OFC measurements had an excellent agreement with manual measurements, with a very strong regression score (R2 = 0.969) and a small mean difference of -0.1 cm (-0.2%). The limits of agreement ranged from -0.93 to 0.74cm, falling within the reported limits of agreement for manual OFC measurements. High interrater and intrarater reliability of OFC, cephalic index, and volume measurements were also demonstrated. The proposed method for automated OFC measurements was found to be a reliable alternative to manual measurements, which may be particularly beneficial in young children who undergo 3D imaging in craniofacial centers as part of their treatment protocol and in research settings that require a reproducible and transparent pipeline for anthropometric measurements. The method has been incorporated into CraniumPy, an open-source tool for 3D image visualization, registration, and optimization, which is publicly available on GitHub (https://github.com/T-AbdelAlim/CraniumPy).

Solid validity evidence for two tools assessing competences in musculoskeletal ultrasound: a validity study.

Musculoskeletal ultrasound (MSUS) is increasingly used by rheumatologists in daily clinical practice. However, MSUS is only valuable in trained hands, and assessment of trainee competences is therefore essential before independent practice. Thus, this study aimed to establish validity evidence for the EULAR and the Objective Structured Assessment of Ultrasound Skills (OSAUS) tools used for assessing MSUS competences. Thirty physicians with different levels of MSUS experience (novices, intermediates, and experienced) performed four MSUS examinations of different joint areas on the same rheumatoid arthritis patient. All examinations were video recorded (n = 120), anonymized, and subsequently assessed in random order by two blinded raters using first the OSAUS assessment tool followed by the EULAR tool 1 month after. The inter-rater reliability between the two raters was high for both the OSAUS and EULAR tools, with a Pearson correlation coefficient (PCC) of 0.807 and 0.848, respectively. Both tools demonstrated excellent inter-case reliability, with a Cronbach's alpha of 0.970 for OSAUS and 0.964 for EULAR. Furthermore, there was a strong linear correlation between the OSAUS and the EULAR performance scores and the participants' experience levels (R2 = 0.897 and R2 = 0.868, respectively) and a significant discrimination between different MSUS experience levels (P < 0.001 for both). MSUS operator competences can be assessed reliably and valid using either the OSAUS or the EULAR assessment tool, thereby allowing a uniform competency-based MSUS education in the future. Although both tools demonstrated high inter-rater reliability, the EULAR tool was superior to OSAUS. ClinicalTrials.gov, http://clinicaltrials.gov, NCT05256355.

Comparison of Different Methods of Measuring Finger Range of Motion via Telehealth

This study examined the accuracy and reliability of measuring total motion of the fingers via telehealth using the following three different methods: (1) goniometry, (2) visual estimation, and (3) electronic protractor. Measurements were compared with in-person measurement, which was assumed to be the reference standard. Thirty clinicians measured finger range of motion from prerecorded videos of a mannequin hand with articulating fingers, which was posed in extension and flexion that simulated a telehealth visit, using a goniometer with results blinded to the clinician (blinded goniometry), visual estimation, and an electronic protractor, in random order. Total motion was calculated for each finger and for all four fingers in sum. The experience level, familiarity with measuring finger range of motion, and opinions of measurement difficulty were assessed. Measurement with the electronic protractor was the only method equivalent to the reference standard within 20°. Remote goniometer and visual estimation did not fall within the acceptable error margin of equivalence, and both underestimated total motion. Electronic protractor also had the highest interrater reliability (intraclass correlation [upper limit, lower limit], 0.95 [0.92, 0.95]); goniometry (intraclass correlation, 0.94 [0.91, 0.97]) was nearly identical, whereas visual estimation (intraclass correlation, 0.82 [0.74, 0.89]) was much lower. Clinicians' experience and familiarity with range of motion measurements had no relationship with the findings. Clinicians reported visual estimation as the most difficult (80%) and electronic protractor as the easiest method (73%). This study showed that traditional in-person forms of measurement underestimate finger range of motion via telehealth; a new computer-based method (ie, electronic protractor) was found to be more accurate. The use of an electronic protractor can be beneficial to clinicians measuring range of motion in patients virtually.

0795 Implementation of the Pediatric Craniofacial Screening Tool for Sleep Disordered Breathing (PCSS): An Observational Pilot Study

Abstract Introduction Pediatric sleep-disordered breathing (SDB) is highly prevalent and associated with craniofacial risk factors not routinely assessed. We developed the Pediatric Craniofacial Screening Tool for Sleep (PCSS) to assess which craniofacial risk factors are closely correlated with SDB symptoms in children. Methods Pilot study of children ages 5-12y recruited from eight dental sites across the country to participate in the PCSS screening by dental providers. The primary outcome was SDB measured by the validated PSQ-SRBD questionnaire (22 items, range 0-1, and &amp;gt;0.33 signifying increased SDB risk). The predictors included PCSS total score and individual items comprising of 21 craniofacial features. Neighborhood socioeconomic status (NSES) was characterized by using the Area Deprivation Index (ADI). Correlation and regression analyses modeled the association between PSQ-SRBD and total PCSS score. To assess inter-rater reliability, a subset of subjects had a second PCSS screen by a second dental provider. Results The sample included 97 children (94%-Non-Hispanic-White, 4% Hispanic, 2% other; 49%-female), with a mean age of 8.6±2.2. PSQ-SRBD scores were elevated in 20 (20.6%) participants. Three children were reported to overweight (on item specific question of PSQ-SRBD scale). Children with elevated PSQ-SRBD (&amp;gt;0.33) had lower NSES compared to children with PSQ-SRBD scores &amp;lt; 0.33. PSQ-SRBD was associated with total PCSS score (OR=1.17 CI[1.01-1.36], p=0.033) and individual features including, narrow vaulted palate (OR=3.22, CI[0.94-10.93], p=0.03), forward head posture (OR=3.13, CI[1.11, 9.30], p=0.03), tongue thrust (OR=7.04, CI[1.20, 47.67], p=0.03), tongue tie (OR=7.58, CI[1.88, 33.58], p=0.005) and heart-shaped tongue (OR=11.21, CI[1.81, 98.11], p=0.01) after adjustment with age, sex, race and ethnicity. The correlation between total PCSS score and PSQ-SRBD was low (R=0.39, p= &amp;lt; 0.01), however in subjects with at least one positive craniofacial feature, the correlation between significant PCSS features and PSQ-SRBD was moderate (R=0.54, p= &amp;lt; 0.01). There was high inter-rater reliability (kappa=0.75) in a subset of 53 subjects with second PCSS screens. Conclusion Multiple craniofacial features assessed during routine dental visits were associated with increased risk for pediatric SDB. This study points to the utility of a craniofacial scale to assess SDB risk in children. This pilot study will be expanded to include a more diverse patient population. Support (if any)

Determinants of point-of-care ultrasound lung sliding amplitude in mechanically ventilated patients

BackgroundAlthough lung sliding seen by point-of-care ultrasound (POCUS) is known to be affected to varying degrees by different physiologic and pathologic processes, it is typically only reported qualitatively in the critical care setting. Lung sliding amplitude quantitatively expresses the amount of pleural movement seen by POCUS but its determinants in mechanically ventilated patients are largely unknown.MethodsThis was a single-center, prospective, observational pilot study examining 40 hemithoraces in 20 adult patients receiving mechanical ventilation. Each subject had lung sliding amplitude measured in both B-mode and by pulsed wave Doppler at their bilateral lung apices and bases. Differences in lung sliding amplitude were correlated with anatomical location (apex vs base) as well as physiologic parameters including positive end expiratory pressure (PEEP), driving pressure, tidal volume and the ratio of arterial partial pressure of oxygen (PaO2) to fraction of inspired oxygen (FiO2).ResultsPOCUS lung sliding amplitude was significantly lower at the lung apex compared to the lung base in both B-mode (3.6 ± 2.0 mm vs 8.6 ± 4.3 mm; p < 0.001) and the pulsed wave Doppler mode (10.3 ± 4.6 cm/s vs 13.9 ± 5.5 cm/s; p < 0.001) corresponding to expected distribution of ventilation to the lung bases. Inter-rater reliability of B-mode measurements was excellent (ICC = 0.91) and distance traversed in B-mode had a significant positive correlation with pleural line velocity (r2 = 0.32; p < 0.001). There was a non-statistically significant trend towards lower lung sliding amplitude for PEEP ≥ 10 cmH2O, as well as for driving pressure ≥ 15 cmH2O in both ultrasound modes.ConclusionPOCUS lung sliding amplitude was significantly lower at the lung apex than the lung base in mechanically ventilated patients. This was true when using both B-mode and pulsed wave Doppler. Lung sliding amplitude did not correlate with PEEP, driving pressure, tidal volume or PaO2:FiO2 ratio. Our findings suggest that lung sliding amplitude can be quantified in mechanically ventilated patients in a physiologically predictable way and with high inter-rater reliability. A better understanding of POCUS derived lung sliding amplitude and its determinants may aid in the more accurate diagnosis of lung pathologies, including pneumothorax, and could serve as a means of further reducing radiation exposure and improving outcomes in critically ill patients.

Validation of myocarditis diagnoses in the Swedish patient register for analyses of potential adverse reactions to COVID-19 vaccines.

Coronavirus disease 2019 (COVID-19) mRNA vaccines are associated with an increased risk of myocarditis using hospital discharge diagnoses as an outcome. The validity of these register-based diagnoses is uncertain. Patient records for subjects < 40 years of age and a diagnosis of myocarditis in the Swedish National Patient Register were manually reviewed. Brighton Collaboration diagnosis criteria for myocarditis were applied based on patient history, clinical examination, laboratory data, electrocardiograms, echocardiography, magnetic resonance imaging and myocardial biopsy. Poisson regression was used to estimate incidence rate ratios, comparing the register-based outcome variable to validated outcomes. Interrater reliability was assessed by a blinded re-evaluation. Overall, 95.6% (327/342) of cases registered as myocarditis were confirmed (definite, probable or possible myocarditis according to Brighton Collaboration diagnosis criteria, positive predictive value 0.96 [95% CI 0.93-0.98]). Of the 4.4% (15/342) cases reclassified as no myocarditis or as insufficient information, two cases had been exposed to the COVID-19 vaccine no more than 28 days before the myocarditis diagnosis, two cases were exposed >28 days before admission and 11 cases were unexposed to the vaccine. The reclassification had only minor impact on incidence rate ratios for myocarditis following COVID-19 vaccination. In total, 51 cases were sampled for a blinded re-evaluation. Of the 30 randomly sampled cases initially classified as either definite or probably myocarditis, none were re-classified after re-evaluation. Of the in all 15 cases initially classified as no myocarditis or insufficient information, 7 were after re-evaluation re-classified as probable or possible myocarditis. This re-classification was mostly due to substantial variability in electrocardiogram interpretation. This validation of register-based diagnoses of myocarditis by manual patient record review confirmed the register diagnosis in 96% of cases and had high interrater reliability. Reclassification had only a minor impact on the incidence rate ratios for myocarditis following COVID-19 vaccination.

Translation of soundscape perceptual attributes from English to Turkish

The International Standard Organization (ISO) published the standard series on soundscape for the identification, data collection and data analysis. However, since all these standards are in English language, the reliable standardized usage in other languages and its applicability is questionable. Thus, this two-staged study aims: i) to determine the Turkish equivalences of the 8 soundscape perceptual attributes that are published in ISO/TS 12913–2:2018 and ISO/TS 12913–3:2019, ii) to analyze if the determined Turkish attributes have concept equivalence to the original ones, and iii) to determine if the translated Turkish scale is reproducible. The first stage involved the translation of the attributes by focus group discussions and finalization by Turkish linguistic experts. As a result, the attributes ‘eventful’, ‘vibrant’, ‘pleasant’, ‘calm’, ‘uneventful’, ‘monotonous’, ‘annoying’, and ‘chaotic’ are translated to Turkish as ‘hareketli’, ‘coşkulu’, ‘keyifli’, ‘sakin’, ‘durağan’, ‘tekdüze’, ‘rahatsız edici’, and ‘karmaşık’, respectively. The second stage involves the analysis of reproducibility in terms of inter-rater reliability and conceptual validity. It is found that the Turkish scale is reproducible based on high inter-rater reliability in all attributes. Context validity at a conceptual level is analyzed both in terms of the difference between the average scores given to the English attributes and their corresponding Turkish equivalences and the correlation between the English and Turkish scores given to each attribute. The highest difference between the average scores (around 10 points on a slider scale of 0 to 100) is found to be in the translation of ‘vibrant’ while the lowest correlated one (slightly lower than 0.5) is found in ‘chaotic’ attribute as in line with literature. Despite this result, when the scores are reduced to 2 dimensions as pleasantness and eventfulness, it is seen that there is a high correlation between the English and Turkish scales. It is considered that the results obtained from this research could act as a base in the future for the establishment of Turkish Standards on soundscape and standardization of the translated and validated Turkish soundscape perceptual attributes and the ‘perceived affective quality’ scale defined under ISO/TS 12913–2:2018 in English.

Assessing engagement in rehabilitation: development, validity, reliability, and responsiveness to change of the Rehabilitation Observation Measure of Engagement (ROME)

Purpose We describe the development of an observational video coding tool, the Rehabilitation Observation Measure of Engagement (ROME), to quantify engagement in rehabilitative settings at the person (internal state of an individual) and between-system (interaction between individuals) level. Methods Forty-nine children with unilateral spastic cerebral palsy (29 males; Age: M = 9.28 yrs, SD = 3.08 yrs) and their interventionists were videotaped during different activities. Construct validity was examined by correlating the ROME with the Engagement vs. Disaffection with Learning Survey and the Pediatric Rehabilitation Intervention Measure of Engagement – Observation questionnaire. Inter- and intra-rater reliability were examined using two independent raters. The ROME’s responsiveness to change was examined by comparing scores across activities. Results For construct validity, results showed a positive correlation for person-level engagement (r = 0.444, p = 0.003). No relationship was found between-system-level engagement. High intrarater (91.8%) and interrater (96.1%) reliability was found. The ROME’s responsiveness to change was supported by children exhibiting lower engagement scores during repetitive shaping activities. Conclusion These findings provide evidence that the ROME is a reliable tool to objectively examine the construct of engagement within rehabilitation and is valid for quantifying person-level engagement. It provides information that cannot be extracted from questionnaires and can help guide intervention decisions.

Skill-Mix Changes Targeting Health Promotion and Prevention Interventions and Effects on Outcomes in all Settings (Except Hospitals): Overview of Reviews.

Objectives: Skill-mix changes to step up health promotion and prevention are increasing, but there is limited evidence on their effects. Methods: Overview of reviews, based on a protocol. The search was carried out in six databases, screening was performed ensuring high interrater reliability. All countries, health professions and lay workers in all settings (except hospitals) were included, quality appraisals performed. Results: A total of 31 systematic reviews were included. Expanded roles performing outreach (e.g., home visits) had mostly positive effects on access and health outcomes, primarily for hard-to-reach groups. Task-shifting in colorectal or skin cancer screenings (performed by advanced practice nurses) were suggested effective; supporting roles (by community health workers) increased uptake in screenings, but based on limited evidence. Expanded roles of various professions focusing on lifestyle modification showed promising effects in most reviews, including weight, diet, smoking cessation and physical activity. Reviews on cost-effectiveness were based on limited evidence. Conclusion: Promising skill-mix changes included expanded roles providing lifestyle modifying interventions, task-shifting, and outreach roles for hard-to-reach groups, whereas evidence on costs was limited.

The Importance of Patient-Focused Drug Development in Pemphigus and Pemphigoid

The Importance of Patient-Focused Drug Development in Pemphigus and Pemphigoid