Strong Inter-rater Reliability Research Articles

Computational Metrics Can Provide Quantitative Values to Characterize Arthroscopic Field of View

PurposeThe purpose of this study was to determine the inter-rater reliability of arthroscopic video quality, determine correlation between surgeon rating and computational image metrics, and facilitate a quantitative methodology for assessing video quality.MethodsFive orthopaedic surgeons reviewed 60 clips from deidentified arthroscopic shoulder videos and rated each on a four-point Likert scale from poor to excellent view. The videos were randomized, and the process was completed a total of three times. Each user rating was averaged to provide a user rating per clip. Each video frame was processed to calculate brightness, local contrast, redness (used to represent bleeding), and image entropy. Each metric was then averaged over each frame per video clip, providing four image quality metrics per clip.ResultsInter-rater reliability for grading video quality had an intraclass correlation of .974. Improved image quality rating was positively correlated with increased entropy (.8142; P < .001), contrast (.8013; P < .001), and brightness (.6120; P < .001), and negatively correlated with redness (−.8626; P < .001). A multiple linear regression model was calculated with the image metrics used as predictors for the image quality ranking, with an R-squared value of .775 and root mean square error of .42.ConclusionsOur study demonstrates strong inter-rater reliability between surgeons when describing image quality and strong correlations between image quality and the computed image metrics. A model based on these metrics enables automatic quantification of image quality.Clinical RelevanceVideo quality during arthroscopic cases can impact the ease and duration of the case which could contribute to swelling and complication risk. This pilot study provides a quantitative method to assess video quality. Future works can objectively determine factors that affect visualization during arthroscopy and identify options for improvement.

Feedback Methods in an Interprofessional Mock Paging Program.

Many medical schools offer a culminating internship readiness experience. Curricula focus on particular knowledge and skills critical to internship, such as answering urgent nursing pages. Studies have shown student performance improvement with mock paging education programs, but the role of feedback versus self-regulated practice has not been studied. The interprofessional mock paging program included 156 medical students enrolled in a 4th-year internship readiness course and 44 master's level direct entry nursing students. Medical students were randomized to receive verbal feedback immediately after each of the three phone calls (intervention group) or delayed written feedback (control group) after the third phone call only. Specialty-specific case scenarios were developed and a single checklist for all scenarios was developed using the communication tool ISBAR. Medical students and nursing students had separate training sessions before the pages commenced. The nursing students administered the phone calls and evaluated the medical students by ISBAR checklist. An interrater reliability measure was obtained with physician observation of a selection of phone calls. After adjusting for the case effects (different case scenarios for different specialties), students showed no statistically significant differences on checklist scores for case 1 (first case, F = 1.491, df = 1, p = .224), but did show statistically significant differences on checklist scores for case 3 (final case, F = 12.238, df = 1, p = .001). Strong interrater reliability was found between the faculty physician and observed nursing students (ICC = .89). Immediate feedback significantly improves student checklist scores with a mock paging program. This finding suggests that coaching with feedback may have advantages above self-regulated learning.

Assessing the disproportionality of depressive reactions to life stress.

Prior to DSM-5, depression had long been conceptualized as symptoms without cause or disproportionate to one's circumstances. A central challenge to considering whether a depressive reaction is disproportionate is the lack of measures assessing disproportionality. Drawing on a study of patients participating in cognitive behavioural therapy for depression, we evaluate two new measures of the disproportionality of one's depressive symptoms to their recent life circumstances. To assess the disproportionality of depressive symptoms, we developed an interview-based assessment and a brief self-report measure. We employed both assessments in a sample of 126 patients who participated in cognitive behavioural therapy for depression. Initial evidence for the reliability and validity of both self-report and interview-based approaches appeared promising. Interview judges demonstrated strong inter-rater reliability on life stress ratings, and both forms of disproportionality showed a pattern of correlations with variables reflecting greater clinical complexity, including self-reported personality dysfunction and symptom severity. Comorbid generalized anxiety disorder (GAD) was the only previously proposed marker of complicated depression to significantly predict both disproportionality scores. The sample had limited ethnic diversity and tended to be highly educated. Participants all met criteria for major depressive disorder and sought treatment. Two assessments of the disproportionality of depressive reactions demonstrated evidence of reliability and validity. Despite limited efforts at assessment to date, these results suggest these measures may be able to reliably distinguish the disproportionality of depressive reactions and allow clinicians to better assess contextual life stress depressive disorders.

Systematic administration and analysis of verbal fluency tasks: Preliminary evidence for reliable exploration of processes underlying task performance

Verbal fluency (VF) tasks are typically scored by the number of acceptable words generated within an allotted time (i.e., total score). However, total scores do not provide insight into verbal and executive processes underlying VF task performance. Further analyses have been implemented to increase the analytical power of VF tasks, but systematic scoring guidelines are needed. We generated instructions for administration, scoring, and analyses of total scores, errors, temporal parameters, clustering, and switching with strong inter-rater reliability. To investigate the reliability of the proposed analysis, we modeled the performance of Finnish-speaking older adults (N = 50) in phonemic (/k/, /a/, and /p/) and semantic (animals) categories. Our results are in line with previous studies: We observed a higher performance on semantic than phonemic fluency (p ≤ 0.001, d = 0.91) and significant effects for education (p ≤ 0.001, d = 1.11) and gender (p ≤ 0.001, d = −1.11), but not for age (p = 0.10, d = 0.48). Most errors were repetitions. Performance declined over the allotted time frame as measured in 15-s segments (all ps < 0.001 with medium to large effect sizes). Task congruent clustering and switching were productive strategies (all ps < 0.001 with large effect sizes), and participants generated task discrepant clusters in both phonemic (p = 0.004, d = 0.69) and semantic tasks (p = 0.66, d = 0.18). The results substantiate the proposed method, providing evidence that these guidelines are a reliable starting point for VF task performance analyses in various clinical populations investigating VF task performance in depth.

Strong Agreement Between Magnetic Resonance Imaging and Radiographs for Caton–Deschamps Index in Patients With Patellofemoral Instability

PurposeTo compare the measurements of the Caton–Deschamps index on preoperative magnetic resonance imaging and radiographs of patients undergoing operative management of patellar instability.MethodsPatients who underwent primary medial patellofemoral ligament reconstruction and/or tibial tubercle osteotomy between January 2015 and November 2019 were assessed. Caton–Deschamps indices were measured by 3 independent reviewers on both radiographs and magnetic resonance imaging. Intra- and interclass correlation coefficients and a Bland–Altman analysis were calculated to assess inter-rater reliability and measurement agreement between radiographic and magnetic resonance imaging.ResultsSeventy-two patients (73 knees) were identified. The average Caton–Deschamps index was 1.23 ± 0.18 on radiograph and 1.26 ± 0.18 on magnetic resonance imaging. Strong inter-rater reliability was observed between reviewers for both radiographic and magnetic resonance imaging Caton–Deschamps indices (intraclass correlation coefficients 0.700 and 0.715, respectively). Pooled observer measurements revealed a moderate agreement between radiographic and magnetic resonance imaging for patella to tibia distance, weak agreement for patellar articular cartilage distance, and strong agreement for the Caton–Deschamps index (intraclass correlation coefficients 0.687, 0.485, and 0.749, respectively). Bland–Altman analysis demonstrated a mean difference in Caton–Deschamps index of –0.03 ± 0.15 (95% limits of agreement: –0.29 to 0.23) between radiographic and magnetic resonance imaging, meaning that Caton–Deschamps indices were on average 0.03 lower on radiographic than on magnetic resonance imaging.ConclusionsThe Caton–Deschamps index has strong agreement between radiographic and magnetic resonance imaging in patients undergoing patellar stabilization surgery. Either modality can be reliably used to preoperatively assess patellar height.Level of EvidenceLevel IV, diagnostic case series.

Towards construct validity of relational aggression: An examination of the Children's Social Behavior Scale.

Relational aggression-or behavior intended to harm the relationships of its victims-has been the focus of interdisciplinary study across developmental, clinical, personality, and social psychology in the last several decades. One of the primary measures used to assess relational aggression in youth is the Children's Social Behavior Scale (CSBS; Crick & Grotpeter, 1995), but despite its common usage, the construct validity of this measure has not been comprehensively assessed. In the present study, we used a multistage construct validity framework to thoroughly investigate the nature of relational aggression across six community samples totaling 3,102 youth and their caregivers. We used multiple methods to map the reliability, internal or structural validity, and external validity of this scale. Through these analyses, we found that CSBS Relational Aggression demonstrated strong internal consistency, test-retest, and interrater reliability as well as a robust single factor structure and invariance across multiple demographic groups. External validity analyses positioned relational aggression within a theoretically consistent nomological net including psychopathology, personality, and social developmental factors. Contrary to concerns about the validity of self- and parent-reports of relational aggression, both parent- and youth-report forms of the CSBS Relational Aggression scale demonstrated strong reliability and validity. While construct validation has received inadequate attention in the psychological literature to date, through this project, we aimed to demonstrate how this approach may be used to investigate existing measures across psychological research. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

ICD-11 Guidelines for Mental and Behavioral Disorders of Children and Adolescents: Reliability and Clinical Utility.

ICD-11 clinical guidelines for mental and behavioral disorders must be tested in clinical settings to guarantee their usefulness worldwide. The purpose of this study was to evaluate interrater reliability and clinical utility of the ICD-11 guidelines for children and adolescents in assessing and diagnosing mood, anxiety, and fear-related disorders; attention-deficit hyperactivity disorder (ADHD); and disruptive behavioral disorder (DBD). Children and adolescents ages 6-17 from two specialized settings in Mexico City were interviewed. Each was interviewed by a pair of psychiatrists (interviewer and observer), who independently codified established diagnoses and evaluated the clinical utility of the guidelines with each participant. Kappa values were calculated to determine the level of general diagnostic correlation between the two clinicians. A total of 25 psychiatrists evaluated 52 children and adolescents. Kappa values between clinicians ranged from 0.46 to 0.53 for mood, anxiety, and fear-related disorders and for ADHD; the kappa value was 0.81 for DBD guidelines. Over 80% of psychiatrists reported that the guidelines, qualifiers, and descriptions of developmental presentations were quite useful. ICD-11 guidelines for mental and behavioral disorders of children and adolescents demonstrated mostly moderate interrater reliability and strong interrater reliability in the case of DBD. A large proportion of clinicians regarded the guidelines as quite useful clinical tools.

Psychometrics of the Pragmatic Rating Scale for School-Age Children With a Range of Linguistic and Social Communication Skills.

Purpose Social communication or pragmatic skills are continuously distributed in the general population. Impairment in these skills is associated with two clinical disorders, autism spectrum disorder (ASD) and social (pragmatic) communication disorder. Such impairment can impact a child's peer acceptance, school performance, and current and later mental health. Valid, reliable, examiner-rated observational measures of social communication from a semistructured language sample are needed to detect social communication impairment. We evaluated the psychometrics of an examiner-rated measure of social (pragmatic) communication, the Pragmatic Rating Scale-School Age (PRS-SA). Method The analytic sample consisted of 130 children, ages 7-12 years, from five mutually exclusive groups: ASD (n = 25), language concern (LC; n = 5), ASD + LC (n = 10), social communication impairment only (n = 22), and typically developing (TD; n = 68). All children received language and autism assessments. The PRS-SA was rated separately using video-recorded communication samples from the Autism Diagnostic Observation Schedule. Assessment data were employed to evaluate the psychometrics of the PRS-SA. Analysis of covariance models were used to assess whether the PRS-SA would detect differences in social communication functioning across the five groups. Results The PRS-SA demonstrated strong internal reliability, concurrent validity, and interrater reliability. PRS-SA scores were significantly higher in all groups compared to the TD group and differed significantly in most pairwise comparisons; the ASD + LC group had the highest (more atypical) scores. Conclusions The PRS-SA shows promise as a measure of social communication skills in school-age verbally fluent children with a range of social and language abilities. More research is needed with a larger sample, including a wider age range and geographical diversity, to replicate findings. Supplemental Material https://doi.org/10.23641/asha.15138240.

A preliminary validation of a dynamic speech motor assessment for Swedish-speaking children with childhood apraxia of speech

Objective Children with speech sound disorders (SSDs) have difficulties affecting different levels of speech production. For treatment to be beneficial, it is important to differentiate between Childhood Apraxia of Speech (CAS) – a motor speech disorder with deficits in speech praxis – and other SSDs (nonCAS-SSD). We have previously developed a motor speech examination Dynamisk motorisk talbedömning (DYMTA). We aimed to evaluate DYMTAs reliability and validity in a small-scale sample to estimate DYMTAs usability in diagnostic settings. Methods Speech, language, and oral motor abilities were assessed in 45 children between 40 and 106 months. Intra- and inter-rater reliability of DYMTA were analyzed. Further, DYMTAs ability to validly discriminate between children with CAS and nonCAS-SSD was assessed. Results The intra-rater reliability for the scores of DYMTA was strong, with ICCs ranging from 0.97 to 1.0. DYMTA total score had strong inter-rater reliability as evidenced both by the agreement estimates (DYMTA-A: 0.91 and DYMTA-B: 0.87) and the ICCs (0.97 and 0.96). Inter-rater reliability was also strong for the separate subscores on agreement estimates and for all subscores on ICCs, except for the Prosody subscores. DYMTA accurately discriminated between children with CAS and nonCAS-SSD in this small sample with an AUC of 0.92 for DYMTA-A and 0.94 for DYMTA-B. Conclusions With its focus on speech movements, DYMTA could serve as a valuable addition to other tests when assessing children’s speech motor performance. This first examination suggests that DYMTA may be both a reliable and valid tool in the diagnostic process of SSD.

Development and psychometric evaluation of the Test of Practical Judgment alternate form (Form B)

The Test of Practical Judgment (TOP-J) is increasingly used by neuropsychologists to measure everyday judgment ability in older adulthood. In the present study, we developed an alternate TOP-J Form B, which may be used to reduce practice effects for repeat assessment situations or in place of the original Form A. In developing the measure, special attention was given to limiting cultural bias and making items similar in content and difficulty to Form A. The TOP-J Form B was piloted in a clinical geriatric sample (N = 77) in the Midwestern U.S. Subsequently, older adults (N = 130) were recruited from several boroughs of New York City and surrounding areas (mean age = 77; mean years of education = 16; 69% female; 28% Black/African-American, 11% Hispanic). In this validation sample, both the 9-item and 15-item versions of the TOP-J Form B showed strong psychometric properties, including good unidimensional model fit in confirmatory factor analysis, preliminary convergent/divergent and criterion validity evidence, and strong inter-rater reliability, ICC (2, 1) = .93. The means and standard deviations for the TOP-J Form A and Form B were highly similar, particularly for the 9-item forms in which there was less than a one-point mean difference. Preliminary normative data for cognitively intact participants (n = 73) were established. We present means and standard deviations that will allow for the calculation of z scores as Form B scores were normally distributed. The newly developed TOP-J Form B should be useful in diverse clinical and research settings.

Measuring parent strategy use in early intervention: Reliability and validity of the Naturalistic Developmental Behavioral Intervention Fidelity Rating Scale across strategy types.

Children with autism spectrum disorder benefit from early intervention to improve social communication, and parent-implemented interventions are a feasible and family-centered way to increase the amount of treatment they receive. For these treatments to be effective, it is important for the parent to implement the strategies as intended. However, measurement of parent strategy use is inconsistent across studies of parent-implemented interventions. This study evaluates the quality of the NDBI-Fi, an efficient measure, compared to a more time-consuming measure that is known to be precise. Videos of parents playing with their children were used to compare these two measurement methods. Results demonstrated that the NDBI-Fi was of good quality: scorers had high levels of agreement, the NDBI-Fi was similar to the more precise measure in rating parents after intervention, it detected changes from before to after intervention, and it detected differences when parents learned different types of intervention strategies. The NDBI-Fi was not as precise as the other measure across all strategies before parents learned intervention. Taken together, the findings of this study support the use of the NDBI-Fi as a high-quality outcome measure.

A perceptual outcome measure of velopharyngeal function based on the Cleft Audit Protocol for Speech-Augmented (CAPS-A VPC-Sum): Validation through a speech osteotomy study.

The status of the velopharyngeal mechanism can be inferred from perceptual ratings of specified speech parameters. Several studies have proposed the measure of an overall velopharyngeal composite score based on these perceptual ratings and have reported good validity. The Cleft Audit Protocol for Speech-Augmented (CAPS-A) is a validated and reliable perceptual framework for the assessment of cleft speech and velopharyngeal function used by all Regional Cleft Services in the UK and Ireland. An overall velopharyngeal composite summary score based on the CAPS-A would serve as an important surgical outcome measure of speech. To develop and validate a velopharyngeal composite summary score based on perceptual ratings made on the CAPS-A (CAPS-A VPC-Sum) using data from a maxillary osteotomy (MO) study. There were two surgical groups: a cleft lip and palate (CLP) (N = 20) group and a non-CLP group (N = 10), and a normal control group (N = 20). Participants in groups 1 and 2 were seen for perceptual and instrumental assessments of speech and velopharyngeal function preoperatively (T1), 3 months (T2) and 12 months (T3) postoperatively. Perceptual speech data were collected and rated by independent listeners using CAPS-A. Moderate to strong interrater reliability for perceptual data (rs = 0.503-1.000, all p < 0.01) and strong to very strong reliability for videofluoroscopic measurements (rs = 0.746-0.947) were found. Construct validity of the CAPS-A VPC-Sum was shown by an increase in postoperative scores for the CLP group only ϰ2 (2) = 9.769, p = 0.008 and significant differences between the CLP and the other two groups at T2 and T3 using independent t-tests. Convergent and divergent validity was indicated by a positive moderate correlation with related parameters (e.g., hypernasality rs = 0.869, p < 0.01) and a weak correlation with unrelated parameters (e.g., amount of forward advancement rs = 0.160, p = 0.526). Criterion validity was found by a moderate correlation between closure ratio rs = -0.541, p = 0.020 and CAPS-A VPC-Sum. A velopharyngeal composite score based on perceptually rated parameters serves as an important surgical speech outcome measure. The CAPS-A VPC-Sum is a useful, reliable and valid outcome measure of velopharyngeal function. There are added positive implications for other clinicians using geographically and language-specific adapted versions of the CAPS-A internationally. What is already known on this subject Velopharyngeal composite scores based on perceptually rated speech parameters have been shown to have both clinical and research utility, serving as a useful surgical outcome measure. However, such a composite score must be specifically validated on the perceptual speech framework upon which it is based, as there are differences in measurement methods and terminology across cleft speech perceptual frameworks internationally. What this paper adds to existing knowledge The CAPS-A is a nationally used tool in the UK and Ireland for audit and research purposes with validated and adapted international versions. This paper reports on the validation of the derived velopharyngeal composite score measure based on the CAPS-A and an English-speaking sample, providing evidence of its validity through a speech osteotomy study. What are the potential or actual clinical implications of this work? This work provides CLP teams who use CAPS-A with a validated surgical speech outcome measure of velopharyngeal function. It has positive implications also for adapted versions of the CAPS-A internationally.

Frugal 3D scanning using smartphones provides an accessible framework for capturing the external ear

Three-dimensional (3D) scanning technologies, such as medical imaging and surface scanning, have important applications for capturing patient anatomy to create personalised prosthetics. Digital approaches for capturing anatomical detail as opposed to traditional, invasive impression techniques significantly reduces turnaround times and lower production costs while still maintaining the high aesthetic quality of the end product. While previous case studies utilise expensive 3D scanning and modelling frameworks, their clinical translation is limited due to high equipment costs. In this study, we develop and validate a low-cost framework for clinical 3D scanning of the external ear using photogrammetry and a smartphone camera. We recruited five novice operators who watched an instructional video before scanning 20 healthy adult participant ears who did not have microtia. Our results show that the smartphone-based photogrammetry methodology produces 3D scans of the external ear that were accurate to (1.5±0.4)mm and were (71±14)% complete compared with those from a gold standard reference scanner, with no significant difference observed between operators. A moderate to strong interrater reliability was determined for all novice operators, suggesting that all novice operators were able to capture repeatable scans. The development of this smartphone photogrammetry approach has the potential to provide a non-invasive, inexpensive and accessible means to capture patient morphology for use in clinical assessment and personalised device manufacture, specifically for ear prostheses. We also demonstrate that inexperienced operators can rapidly learn and apply smartphone photogrammetry for accurate and reliable scans of the external ear with important applications for future clinical translation.

The Columbia Behavior Scale for Dementia (CBS-8): Validity and Reliability of a Rapidly Administered New Instrument for Dementia-Related Behaviors in Long-Term Care Settings.

We developed and evaluated the psychometric properties of the Columbia Behavior Scale for Dementia (CBS-8), a rapid instrument that assesses positive symptoms, to enhance behavioral and psychological symptoms of dementia (BPSD) assessment and treatment in long-term care. Psychometric analyses were performed on CBS-8 data from residents (N = 350, age ≥50 years) in 47 Maryland long-term care facilities referred for neurocognitive and mood evaluation. The CBS-8 demonstrated acceptable internal consistency (α = 0.78) and strong interrater reliability (intraclass correlation coefficient = 1.00). CBS-8 scores were correlated with greater cognitive impairment severity (r = -0.34). The diagnosis of dementia with behavioral disturbance had higher CBS-8 scores than other dementia types (e.g., vascular, unspecified) (p < 0.001, η2 = 0.40). Three CBS-8 factors-motor disinhibition, aggression, and psychosis-explained 65% of the variance in overall BPSD. The CBS-8 could enhance BPSD tracking and treatment, strengthen interdisciplinary collaboration, and aid nursing homes in meeting regulations on unnecessary medication use. [Research in Gerontological Nursing, 14(3), 160-168.].

Glenoid Radius of Curvature and Humeral Head Volume Are Associated With Postoperative Dislocation After Arthroscopic Bankart Repair

PurposeTo measure bony morphologic parameters and identify their association with arthroscopic Bankart repair failure.MethodsThis was a retrospective comparative study. The inclusion criteria were primary arthroscopic Bankart repair, no prior shoulder surgery, traumatic cause, and had a Bankart (soft tissue and bony) lesion evident on magnetic resonance imaging (MRI). The exclusion criteria were posterior labral pathology, multidirectional instability, connective tissue disorder, rotator cuff pathology, and those who underwent concomitant shoulder procedures. Patients who had a postoperative redislocation (unstable group) were compared with matched patients who did not (stable group). Data were obtained by chart review and from preoperative MRI. Comparisons were made using the Student t test, Fisher exact test, or χ2 test. Statistical significance was defined as P < .05. Inter-rater reliability was measured between reviewers.ResultsA total of 45 patients experienced a postoperative dislocation and were matched to 90 patients without a postoperative dislocation. There were no differences in demographic and radiographic variables. The radius of curvature (ROC) of the glenoid was larger (shallower) in the unstable group (23.6 mm vs 22.6 mm, P = .05). The humeral head volume (HHV) trended higher in the unstable group (68.9 mL vs 62.9 mL, P = .06). The glenoid volume was not significantly different. A greater percentage of patients with a glenoid ROC of 24.5 mm or greater (62.1% vs 26.4%, P = .0003) and an HHV of 80 mm3 or greater (60.8% vs 28.9%, P = .003) experienced a redislocation compared with patients without these factors. Patients with a glenoid ROC of 24.5 mm or more and an HHV of 80 mm3 or more had greater than 4 times the odds of redislocation (odds ratio, 4.56; 95% confidence interval, 1.44-14.43; P = .0098). Strong inter-rater reliability was found for the HHV, glenoid volume, glenoid ROC, and humeral head ROC measurements (r = 0.94, r = 0.88, r = 0.89, and r = 0.95, respectively).ConclusionsThis study shows that large ROC (shallow) glenoids in conjunction with large humeral heads may predispose patients to failure after arthroscopic Bankart repair.Level of EvidenceLevel III, retrospective comparative trial.

Patellar Apprehension Is Reduced in Most but Not All Patients After Successful Patellar Stabilization

Background: The clinical sign of patellar laxity and the associated symptom of apprehension are mainstays of the physical examination of patellofemoral instability. The apprehension test is widely used as a diagnostic tool and also as an outcome following patellofemoral stabilization surgery. Despite widespread use, the validity, reliability, and responsiveness of the apprehension test have not been established. Purpose: The primary purpose was to evaluate patellar apprehension in patients with recurrent patellofemoral instability to determine if the apprehension test is valid, reliable, and responsive to change after medial patellofemoral ligament (MPFL) reconstruction. The secondary purposes were to concurrently validate patient-rated to surgeon-rated apprehension and to correlate patient-rated apprehension with pathoanatomic characteristics and quality-of-life scores. Study Design: Cohort study (diagnosis); Level of evidence, 3. Methods: A total of 89 patients underwent an MPFL reconstruction and were assessed preoperatively and at 6, 12, and 24 months postoperatively. The patellar apprehension test was performed in neutral extension and 30° of knee flexion. Patient- and surgeon-rated apprehension were graded on a 10-cm visual analog scale (VAS), and the quality of the apprehension symptoms were recorded. Risk factors for patellofemoral instability were documented, and the Banff Patellofemoral Instability Score 2.0 (BPII 2.0) was used to measure disease-specific quality of life. Results: The patient-rated and surgeon-rated apprehension VAS scores were statistically significantly reduced from pre- to postoperatively (P < .001, large effect size) for the neutral extension and 30° of flexion apprehension tests. Postoperatively, over 56% of patients reported a negative apprehension test. Up to 15.7% of patients with positive preoperative apprehension did not demonstrate a reduction postoperatively. Patients consistently graded their apprehension symptoms higher in both quantity and quality than the surgeon. The intraclass correlation coefficient (ICC [2, k]) assessing the preoperative patient and surgeon VAS scores revealed moderate interrater reliability in neutral extension (r = 0.60) and weak interrater reliability in 30° of flexion (r = 0.42). The postoperative ICC (2, k) demonstrated strong interrater reliability for both neutral extension (r = 0.74) and 30° of flexion (r = 0.73). The symptoms of apprehension (physical, emotional, and/or physiological) decreased substantially after surgery. The correlation of postoperative patient-rated apprehension VAS and BPII 2.0 scores demonstrated that less residual patellar apprehension was associated with higher BPII scores in neutral extension (r = −0.35, P = .001). There were no statistically significant correlations revealed between the patient-rated postoperative apprehension VAS and pathoanatomic risk factors. BPII 2.0 scores improved pre- to postoperatively from a mean of 27.6 (SD, 15.7) to 74.3 (SD, 18.3). Three patients (3.4%) sustained a patellar dislocation postoperatively. Conclusion: The patellar apprehension test demonstrated strong validity and responsiveness to change. Interrater reliability ranged from weak to strong. There was a statistically significant reduction in apprehension after patellofemoral stabilization in the majority of patients. Patients graded their apprehension symptoms significantly higher in both quantity and quality than the surgeon. Persistent patellar apprehension after stabilization was correlated with lower quality-of-life scores. No relationship could be found between persistent apprehension and patellofemoral risk factors. These results suggest that use of the apprehension tests as an outcome is inappropriate until further validation is performed.

Going Viral: A Systematic Review of Google Trends in Plastic Surgery and a Recommended Framework for Its Use.

As the leading global search engine with billions of daily queries, Google and its open-source Google Trends (Google, Mountain View, CA) represent an emerging and powerful tool for epidemiological and medical research. Within the field of plastic surgery, Google Trends has yielded insights into online interest for facial feminization surgery, gender-affirmation surgery, cosmetic body procedures, and breast reconstruction, among other common procedures. The existing literature of Google Trends in plastic surgery was systematically reviewed following established Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines. Google Trends' 4 input variables-keyword, region, period, and category-were assessed. Seventeen plastic surgery studies employing Google Trends were reviewed. There was strong inter-rater reliability (Cohen's kappa = 0.68). Analyzing keyword syntax, only 3 of 17 studies (17.6%) used the "+" function to combine terms, which can significantly improve sensitivity. For the region variable, 12 of 16 studies (75%) conducted worldwide searches; yet, none of the studies used any non-English keywords, introducing significant bias. For the period, 88.2% of studies utilized a timespan of greater than 5 years, resulting in monthly intervals between data points in Google Trends. For the "category" variable, none of the studies appear to have employed the "surgery," "cosmetic surgery" or "health" categories to improve specificity. Google Trends is presented as an emerging methodology in plastic surgery research. The strengths and limitations of Google Trends as a resource for plastic surgeons and medical professionals are discussed, and a recommended step-by-step guide for conducting and interpreting Google Trends research is outlined.

Development of the Teaching Interpersonal Skills for Telehealth Checklist.

While interpersonal skills in telehealth may positively impact clinical practice, patient engagement and outcomes, assessment strategies are lacking. We conducted a multi-stage iterative approach to develop and test validity and reliability of the Teaching Interpersonal Skills in Telehealth checklist (TIPS-TC). First, we identified observable communication behaviors from the literature. Second, we surveyed telehealth managers and researchers (N = 11) to rate appropriateness of potential checklist items. Level of agreement (35%-91%) and Kappa statistic (0.18-0.89) confirmed items to be retained and identified items to modify. Based on response patterns and comments, we reduced 44 items to 12 critical checklist behaviors. Third, student clinicians used the checklist with video telehealth consultations and provided feedback. Fourth, we conducted reliability testing with practitioners and administrators (N = 68) who completed the TIPS-TC for two versions of a telehealth scenario. Strong interrater reliability intraclass correlation coefficient (ICC) and test-retest reliability ICC (both p < .001), along with non-significant findings of order effects supported the checklist as an acceptable instrument to differentiate high skill from low skill telehealth sessions. The TIPS-TC offers an evidence-based approach to assessing interpersonal skills in telehealth to help evaluate clinician competence and tailor learning activities across disciplinary roles.

Neuropsychiatric Symptoms Assessment: Cross-cultural Adaptation and Validation of the Portuguese Abe’s BPSD Score (ABS)

ABSTRACT Objectives This study aims to report on the development and psychometric properties of the Portuguese-language Abe’s BPSD score (ABS) to screen for neuropsychiatric symptoms (NPS). Methods ISPOR and COSMIN recommendations were followed to translate and culturally adapt the ABS. A validation study was conducted to assess the psychometric properties of the newly-translated instrument. Outpatients attending a psychogeriatric consultation were included by consecutive referrals and were assessed with the ABS, the Neuropsychiatric Inventory (NPI) and NPI Caregiver Distress scale (NPI-D), and the Mini-Mental State Examination (MMSE). The ABS reliability (internal consistency, item-total correlations, inter-rater and test-retest reliability), validity (concurrent and convergent), feasibility and diagnostic accuracy were examined. Results Overall, 107 participants were included. The ABS Cronbach alpha was 0.672, and item-total correlations ranged from −0.056 to 0.546. Strong inter-rater (ICC 0.997; 95%CI: 0.995–0.999) and test-retest reliability (ICC 0.976; 95%CI: 0.958–0.986) were found. Concurrent validity with NPI was high (rs = 0.847, p < .001), and correlations with MMSE and NPI-D were also significant. An exploratory threshold score ≥2 is proposed to identify clinically relevant NPS. Conclusions Data provide satisfactory proof of ABS psychometric characteristics. Nevertheless, some items exhibited less optimal properties. Clinical Implications The newly-translated instrument proved to be relevant, valid and easy to use in a real geriatric clinical setting.

Psychometric properties of chronic low back pain diagnostic classification systems: a systematic review.

To identify and critically appraise studies evaluating psychometric properties of functionally oriented diagnostic classification systems for Non-Specific Chronic Low Back Pain (NS-CLBP). This review employed methodology consistent with PRISMA guidelines. Electronic databases and journals: (PubMed, EMBASE, Cochrane, PEDro, CINAHL, Index to chiropractic literature, ProQuest, Physical Therapy, Journal of Physiotherapy, Canadian Physiotherapy and Physiotherapy Theory and Practice) were searched from inception until January 2020. Included studies evaluated the validity and reliability of NS-CLBP diagnostic classification systems in adults. Risk of bias was assessed using a Critical Appraisal Tool. Twenty-two studies were eligible: Five investigated inter-rater reliability, and 17 studies analyzed validity of O'Sullivan's classification system (OCS, n = 15), motor control impairment (MCI) test battery (n = 1), and Pain Behavior Assessment (PBA, n = 1). Evidence from multiple low risk of bias studies demonstrates that OCS has moderate to excellent inter-rater reliability (kappa > 0.4). Also, two low risk of bias studies support of OCS-MCI subcategory. Three tests within the MCI test battery show acceptable inter- and intra-rater reliability for clinical use (the "sitting knee extension," the "one leg stance," and the "pelvic tilt" tests). Evidence for the reliability and validity of the PBA is limited to one high bias risk study. Multiple low risk of bias studies demonstrate strong inter-rater reliability for OCS classification specifically OCS-MCI subcategory. Future studies with low risk of bias are needed to evaluate reliability and validity of the MCI test battery and the PBA.