Kappa Range Research Articles

Blood pressure variability assessed by office, home, and ambulatory measurements before and during antihypertensive drug treatment: a sub-analysis of the REVERENT randomized trial.

Blood pressure (BP) variability (BPV) can be assessed using office (OBP), home (HBP), or ambulatory BP (ABP) measurements. This analysis investigated the association and agreement between OBP, HBP, and ABP measurements for BPV assessment at baseline and 10 weeks after initiating antihypertensive drug therapy. Untreated hypertensive patients with elevated BPV were randomized to receive an angiotensin-converting enzyme inhibitor (ramipril) or a calcium channel blocker (nifedipine GITS) in a 10-week, open-label, blinded-end point study. BPV was assessed using standard deviation (SD) and coefficient of variation (CV) (reading-to-reading analyses). Data from 146 participants from three research centers (Athens/Greece; Milan/Italy; Shanghai/China) were analyzed [mean age 53 ± 10 (SD) years, male individuals 60%, baseline systolic OBP, HBP, and 24 h ABP 144 ± 9, 138 ± 10, and 143 ± 10 mmHg, respectively]. Post-treatment minus pre-treatment systolic CV difference was: OBP: 0.3%, P = 0.28; HBP: -0.2%, P = 0.20; 24 h ABP: 1.1%, P < 0.001. Home and ambulatory (not office) BPV indices presented weak-to-moderate correlation, both before and during treatment (range of coefficients 0.04-0.33). The correlation coefficient between systolic HBP and awake ABP CV was 0.21 and 0.28 before and during treatment, respectively ( P < 0.05/< 0.001, respectively). Home and ambulatory (not office) BPV indices presented slight-to-fair agreement (range 64-73%) in detecting participants with high systolic BPV (top quartile of respective distributions) both before and during treatment (kappa range 0.04-0.27). These data showed a weak-to-moderate association between out-of-office (but not office) BPV indices both before and during BP-lowering treatment, with reasonable agreement in detecting individuals with high BPV. Out-of-office BP measurements provide more similar and consistent BPV information than office measurements.

Enhancing Prostate Cancer Diagnosis: Artificial Intelligence–Driven Virtual Biopsy for Optimal Magnetic Resonance Imaging-Targeted Biopsy Approach and Gleason Grading Strategy

An optimal approach to magnetic resonance imaging fusion targeted prostate biopsy (PBx) remains unclear (number of cores, intercore distance, Gleason grading [GG] principle). The aim of this study was to develop a precise pixel-wise segmentation diagnostic artificial intelligence (AI) algorithm for tumor detection and GG as well as an algorithm for virtual prostate biopsy that are used together to systematically investigate and find an optimal approach to targeted PBx. Pixel-wise AI algorithms for tumor detection and GG were developed using a high-quality, manually annotated data set (slides n = 442) after fast-track annotation transfer into segmentation style. To this end, a virtual biopsy algorithm was developed that can perform random biopsies from tumor regions in whole-mount whole-slide images with predefined parameters. A cohort of 115 radical prostatectomy (RP) patient cases with clinically significant, magnetic resonance imaging-visible tumors (n = 121) was used for systematic studies of the optimal biopsy approach. Three expert genitourinary (GU) pathologists (Y.T., A.P., A.Q.) participated in the validation. The tumor detection algorithm (aware version sensitivity/specificity 0.99/0.90, balanced version 0.97/0.97) and GG algorithm (quadratic kappa range vs pathologists 0.77-0.78) perform on par with expert GU pathologists. In total, 65,340 virtual biopsies were performed to study different biopsy approaches with the following results: (1) 4 biopsy cores is the optimal number for a targeted PBx, (2) cumulative GG strategy is superior to using maximal Gleason score for single cores, (3) controlling for minimal intercore distance does not improve the predictive accuracy for the RP Gleason score, (4) using tertiary Gleason pattern principle (for AI tool) in cumulative GG strategy might allow better predictions of final RP Gleason score. The AI algorithm (based on cumulative GG strategy) predicted the RP Gleason score of the tumor better than 2 of the 3 expert GU pathologists. In this study, using an original approach of virtual prostate biopsy on the real cohort of patient cases, we find the optimal approach to the biopsy procedure and the subsequent GG of a targeted PBx. We publicly release 2 large data sets with associated expert pathologists’ GG and our virtual biopsy algorithm.

Concordance of Fathers and Mothers in the Assessment of Their 5-Year-Old Child's Dental Fear.

The aim of this study was to evaluate the concordance of parents' assessments of their child's dental fear. Cross-sectional secondary analysis used data from the multidisciplinary FinnBrain Birth Cohort Study. Child dental fear was assessed at age 5 with the Finnish translation of the modified Children's Fear Survey Schedule Dental Subscale (CFSS-M) by both fathers (n = 588) and mothers (n = 1100). Reply alternatives were from 1 = not afraid to 5 = very afraid and 6 = no experience coded as missing and 1. In total, 514 mother-father pairs were eligible for the analyses. Descriptive statistics, percentage agreement and Cohen's Kappa coefficients were used in the analyses. The concordance of parents' assessments was poor (Kappa range 0.072-0.258). The majority of parents replied "No Experience" to items related to invasive treatment or being unable to breathe. Thus, coding of this reply alternative had a significant impact on the mean values of the child's fear. When assessing the fear of a five-year-old child, it might not be safe to rely only on one parent's assessment, and whether or not the child has experience with the question asked should also be considered.

Development and validation of the international pediatric sleep endoscopy scale (IPSES).

To develop and validate a consensus international pediatric sleep endoscopy scale (IPSES) for pediatric drug-induced sleep endoscopy (DISE). Existing published DISE ratings scales were reviewed in order to develop a consensus rating scale synthesizing the most common features and adding new elements to address areas of controversy. Samples of 30 de-identified DISE video recordings were reviewed to develop and refine the scale. After the consensus scale was defined, a separate sample of 25 de-identified DISE videos were scored with the new consensus scale by the development group and a panel of independent raters. A weighted kappa statistic was used to quantify the inter-rater and intra-rater reliability of the consensus scale at each anatomic level. Among all raters, intra-rater reliability was most variable for the nasal airway (kappa range 0.33-0.94) and best for the lateral oropharynx (kappa range 0.68-0.95). Inter-rater reliability ranged from 0.43 for the nasal airway to 0.57at the soft palate. The IPSES is a reliable consensus scale that reflects the most common features of existing scales and can be adopted as a universal scoring scale for pediatric DISE.

Development and pilot validation of a novel disfigurement severity scale for plexiform neurofibromas in children with neurofibromatosis type 1.

We developed an observer disfigurement severity scale for neurofibroma-related plexiform neurofibromas to assess change in plexiform neurofibroma-related disfigurement and evaluated its feasibility, reliability, and validity. Twenty-eight raters, divided into four cohorts based on neurofibromatosis type 1 familiarity and clinical experience, were shown photographs of children in a clinical trial (NCT01362803) at baseline and 1 year on selumetinib treatment for plexiform neurofibromas (n = 20) and of untreated participants with plexiform neurofibromas (n = 4). Raters, blinded to treatment and timepoint, completed the 0-10 disfigurement severity score for plexiform neurofibroma on each image (0 = not at all disfigured, 10 = very disfigured). Raters evaluated the ease of completing the scale, and a subset repeated the procedure to assess intra-rater reliability. Mean baseline disfigurement severity score for plexiform neurofibroma ratings were similar for the selumetinib group (6.23) and controls (6.38). Mean paired differences between pre- and on-treatment ratings was -1.01 (less disfigurement) in the selumetinib group and 0.09 in the control (p = 0.005). For the disfigurement severity score for plexiform neurofibroma ratings, there was moderate-to-substantial agreement within rater cohorts (weighted kappa range = 0.46-0.66) and agreement between scores of the same raters at repeat sessions (p > 0.05). In the selumetinib group, change in disfigurement severity score for plexiform neurofibroma ratings was moderately correlated with change in plexiform neurofibroma volume with treatment (r = 0.60). This study demonstrates that our observer-rated disfigurement severity score for plexiform neurofibroma was feasible, reliable, and documented improvement in disfigurement in participants with plexiform neurofibroma shrinkage. Prospective studies in larger samples are needed to validate this scale further.

Comparison of Scales for Recording and Evaluating Dental Arcade Shape

Recording scales to evaluate the shape of the dental arcade have been proposed; however, no comparison has been made among different scales to assess which is best. Here, a comparison is made among several scales (Gill 1971, 1995; Gill &amp; Rhine 1986; Hefner &amp; Linde 2018; Hooton, The Harvard Blanks n.d.; Maier 2017; Maier et al. 2015), and they were evaluated on (1) low observer error and (2) strong association with groups. Digital photographs of 659 individuals from collections across the United States were assessed for dental arcade shape. These data were generated by three observers to test for replicability. Additionally, the relationship between scale and sample groups was evaluated using a combination of chi-squares and several measures of effect size (Cramér’s V, Sakoda’s C, Goodman–Kruskal lambda). Values for Fleiss’s kappa range from “fair” to “almost perfect” between intra-and interobserver measures (κ = 0.212–0.851). Nearly all scales exhibit significant associations with the sample groups, though the general trend is toward weak effect sizes. All values for Cramér’s V and Sakoda’s C fall below 0.3, and the lambda statistic does not exceed an average reduction of error of 6%. The Gill scale is the most reliably recorded but is tied to typological approaches to human variation. A five-point scale proposed by Maier (2017) is less replicable but has the largest effect sizes—“moderate” compared to “weak.” Recording the angle of the sides of the dental arcade may be as informative as several of these scales and avoids many typological associations.

Diffusion-weighted imaging complements T2-weighted MRI for tumour response assessment in squamous anal carcinoma

ObjectivesA published tumour regression grade (TRG) score for squamous anal carcinoma treated with definitive chemoradiotherapy based on T2-weighted MRI yields a high proportion of indeterminate responses (TRG-3). We investigate whether the addition of diffusion-weighted imaging (DWI) improves tumour response assessment in the early post treatment period.Materials and methodsThis retrospective observational study included squamous anal carcinoma patients undergoing MRI before and within 3 months of completing chemoradiotherapy from 2009 to 2020. Four independent radiologists (1–20 years’ experience) scored MRI studies using a 5-point TRG system (1 = complete response; 5 = no response) based on T2-weighted sequences alone, and then after a 12-week washout period, using a 5-point DWI-TRG system based on T2-weighted and DWI. Scoring confidence was recorded on a 5-point scale (1 = low; 5 = high) for each reading and compared using the Wilcoxon test. Indeterminate scores (TRG-3) from each reading session were compared using the McNemar test. Interobserver agreement was assessed using kappa statistics.ResultsEighty-five patients were included (mean age, 59 years ± 12 [SD]; 55 women). T2-weighted TRG-3 scores from all readers combined halved from 24% (82/340) to 12% (41/340) with DWI (p < 0.001). TRG-3 scores changed most frequently (41%, 34/82) to DWI-TRG-2 (excellent response). Complete tumour response was recorded clinically in 77/85 patients (91%). Scoring confidence increased using DWI (p < 0.001), with scores of 4 or 5 in 84% (287/340). Interobserver agreement remained fair to moderate (kappa range, 0.28–0.58).ConclusionDWI complements T2-weighted MRI by reducing the number of indeterminate tumour responses (TRG-3). DWI increases radiologist’s scoring confidence.Clinical relevance statementDiffusion-weighted imaging improves T2-weighted tumour response assessment in squamous anal cancer, halving the number of indeterminate responses in the early post treatment period, and increases radiologists’ confidence.Key PointsTumour response based on T2-weighted MRI is often indeterminate in squamous anal carcinoma.Diffusion-weighted imaging alongside T2-weighted MRI halved indeterminate tumour regression grade scores assigned by four radiologists from 24 to 12%.Scoring confidence of expert and non-expert radiologists increased with the inclusion of diffusion-weighted imaging.

Comparing video examinations with physical clinical examinations using finishing pigs with umbilical outpouchings as a model

BackgroundVeterinary telemedicine has only been adopted to some degree. One aspect that needs to be evaluated is clinical examinations using video. The objective of this study was to evaluate agreement between a traditional physical clinical examination and a clinical examination using recorded video using finishing pigs with umbilical outpouchings (umbilical hernias, cysts, and abscesses) as the study unit. A total of 102 finisher pigs with umbilical outpouchings were clinically examined and recorded on video. Four experienced pig veterinarians were allowed to examine each pig for approximately 10 min and were individually asked to fill out a predesigned clinical record. Approximately 1 month after the physical examinations, the veterinarians individually reexamined all 102 pigs in a blinded manner, utilizing the video recordings and filling in a predesigned clinical record.ResultsFor all measurements using a ruler, a high Pearson correlation coefficient was observed between physical and video examinations (range 0.69–0.95). In comparison, the visual bodyweight estimation had a lower Pearson correlation coefficient (range 0.57–0.64). Substantial to almost perfect agreement was observed between the physical and video examinations for abnormal weight distribution on any leg, restricted gait movements, lameness, signs of pain, fitness for transportation, presence of wounds, and categorization of the number of wounds > 4 cm2 on the umbilical outpouching (mean Kappa range 0.67–0.87). Fair agreement was observed for the presence of perineal soiling, ear wounds, pendulation of umbilical outpouching, umbilical outpouching touching the legs, skin not movable over the umbilical outpouching, and umbilical outpouching wound characteristics: type, presence of crusts, active bleeding, thick wound edges, connective tissue (mean Kappa range 0.21–0.40). Slight agreement was observed for umbilical outpouching characteristics: shape, macroscopic vascularization of the skin covering the outpouching, and the presence of scars, excoriations, and fistulas (mean Kappa range 0.10–0.20). Poor agreement was observed for the presence of granulation tissue (mean Kappa = − 0.05).ConclusionsThe agreement between a physical clinical examination and a clinical examination using recorded video of the same pig varies from poor to almost perfect, depending on the clinical sign and the executing veterinarian.

Initial Experience Using Digital Variance Angiography in Context of Prostatic Artery Embolization in Comparison with Digital Subtraction Angiography.

In previous clinical studies digital variance angiography (DVA) provided higher contrast-to-noise ratio (CNR) and better image quality in lower extremity angiography than digital subtraction angiography (DSA). Our aim was to investigate whether DVA has similar quality reserve in prostatic artery embolization (PAE). The secondary aim was to explore the potential advantages of the color-coded DVA (ccDVA) technology in PAE. This retrospective study evaluated 108 angiographic acquisitions from 30 patients (mean ± SD age 68.0 ± 8.9, range 41-87) undergoing PAE between May and October 2020. DSA and DVA images were generated from the same unsubtracted acquisition, and their CNR was calculated. Visual evaluation of DVA and DSA image quality was performed by four experienced interventional radiologists in a randomized, blinded manner. The diagnostic value of DSA and ccDVA images was also evaluated using clinically relevant criteria (visibility of small [< 2.5 mm] and large arteries [> 2.5 mm], feeding arteries and tissue blush) in a paired comparison. Data were analysed by the Wilcoxon signed rank test or the binomial test, the interrater agreement was determined by the Kendall W or Fleiss Kappa analysis. DVA provided 4.11 times higher median CNR than DSA (IQR: 1.72). The visual score of DVA images (4.40 ± 0.05) was significantly higher than that of DSA (3.39 ± 0.07, p < 0.001). The Kendall W analysis showed moderate but significant agreement (WDVA = 0.38, WDSA = 0.53). The preference of ccDVA images was significantly higher in all criteria (63-89%) with an interrater agreement of 58-79%. The Fleiss Kappa range was 0.02-0.18, significant in all criteria except large vessels. Our data show that DVA provides higher CNR and better image quality in PAE. This quality reserve might be used for dose management (reduction of radiation dose and contrast agent volume), and ccDVA technology has also a high potential to assist PAE interventions in the future.

Assessing Gambling Disorder Using Semistructured Interviews or Self-Report? Evaluation of the Structured Clinical Interview for Gambling Disorder Among Swedish Gamblers.

The Structured Clinical Interview for Gambling Disorder (SCI-GD) has the potential to bridge a diagnostic clinical gap, but psychometric evaluations have been scarce, in particular in relation to self-reported diagnostic criteria. This study analyzed existing data, including Swedish gamblers (N = 204) from treatment- and help-seeking contexts, self-help groups, and the general population, who were interviewed with the SCI-GD and completed self-report measures. The results indicated that fewer individuals fulfilled the diagnostic criteria for gambling disorder (GD) with the SCI-GD (n = 110, 54%), compared to a self-report Diagnostic and Statistical Manual of Mental Disorders:5th Edition (DSM-5) questionnaire on GD (n = 145, 71%; p < .001). Agreement between interviews and self-reported criteria was generally low (Fleiss kappa range: 0.31-0.52; r range: 0.35-0.55). A Rasch analysis showed that specific diagnostic criteria varied in difficulty, indicating a general pattern of higher item difficulty for the SCI-GD compared to self-reported DSM-5 criteria. Both the SCI-GD and the self-reported DSM-5 criteria performed well in terms of internal consistency, convergent, and discriminant validity. We conclude that the SCI-GD is a reliable and valid diagnostic tool to assess GD among individuals with various gambling behavior patterns. Further research-related and clinical implications are discussed.

Social risks and social needs in a health insurance exchange sample: a longitudinal evaluation of utilization

BackgroundHealth systems are increasingly attempting to intervene on social adversity as a strategy to improve health care outcomes. To inform health system efforts to screen for social adversity, we sought to explore the stability of social risk and interest in assistance over time and to evaluate whether the social risk was associated with subsequent healthcare utilization.MethodsWe surveyed Kaiser Permanente members receiving subsidies from the healthcare exchange in Southern California to assess their social risk and desire for assistance using the Accountable Health Communities instrument. A subset of initial respondents was randomized to be re-surveyed at either three or six months later.ResultsA total of 228 participants completed the survey at both time points. Social risks were moderate to strongly stable across three and six months (Kappa range = .59-.89); however, social adversity profiles that included participants’ desire for assistance were more labile (3-month Kappa = .52; 95% CI = .41-.64 & 6-month Kappa = .48; 95% CI = .36-.6). Only housing-related social risks were associated with an increase in acute care (emergency, urgent care) six months after initial screening; no other associations between social risk and utilization were observed.ConclusionsThis study suggests that screening for social risk may be appropriate at intervals of six months, or perhaps longer, but that assessing desire for assistance may need to occur more frequently. Housing risks were associated with increases in acute care. Health systems may need to engage in screening and referral to resources to improve overall care and ultimately patient total health.

Rubric development for AI-enabled scoring of three-dimensional constructed-response assessment aligned to NGSS learning progression

IntroductionThe Framework for K-12 Science Education (the Framework) and the Next- Generation Science Standards (NGSS) define three dimensions of science: disciplinary core ideas, scientific and engineering practices, and crosscutting concepts and emphasize the integration of the three dimensions (3D) to reflect deep science understanding. The Framework also emphasizes the importance of using learning progressions (LPs) as roadmaps to guide assessment development. These assessments capable of measuring the integration of NGSS dimensions should probe the ability to explain phenomena and solve problems. This calls for the development of constructed response (CR) or open-ended assessments despite being expensive to score. Artificial intelligence (AI) technology such as machine learning (ML)-based approaches have been utilized to score and provide feedback on open-ended NGSS assessments aligned to LPs. ML approaches can use classifications resulting from holistic and analytic coding schemes for scoring short CR assessments. Analytic rubrics have been shown to be easier to evaluate for the validity of ML-based scores with respect to LP levels. However, a possible drawback of using analytic rubrics for NGSS-aligned CR assessments is the potential for oversimplification of integrated ideas. Here we describe how to deconstruct a 3D holistic rubric for CR assessments probing the levels of an NGSS-aligned LP for high school physical sciences.MethodsWe deconstruct this rubric into seven analytic categories to preserve the 3D nature of the rubric and its result scores and provide subsequent combinations of categories to LP levels.ResultsThe resulting analytic rubric had excellent human- human inter-rater reliability across seven categories (Cohen’s kappa range 0.82–0.97). We found overall scores of responses using the combination of analytic rubric very closely agreed with scores assigned using a holistic rubric (99% agreement), suggesting the 3D natures of the rubric and scores were maintained. We found differing levels of agreement between ML models using analytic rubric scores and human-assigned scores. ML models for categories with a low number of positive cases displayed the lowest level of agreement.DiscussionWe discuss these differences in bin performance and discuss the implications and further applications for this rubric deconstruction approach.

Physician Use of Multiple Criteria to Diagnose Periprosthetic Joint Infection May Be Less Accurate Than the Use of an Individual Test.

Introduction Multiple-criterion scoring systems for periprosthetic joint infection (PJI) can be algorithmically implemented in research, diagnostically outperforming individual tests. This improved performance may be lost in the practice setting, where clinicians rarely utilize strict algorithms. The ability of physicians to interpret multiple criteria for PJI and confront the complexity of combining them into a final diagnosis has never been studied. This study assessed the diagnostic characteristics of physicians using multiple criteria to diagnose PJI and compared the physicians' diagnostic accuracy to that of individual tests. Methods A total of 12 physicians, including academic arthroplasty surgeons (N=4), community arthroplasty surgeons (N=4), and infectious disease (ID) specialists (N=4) were asked to use their routine clinical diagnostic practice to assign a diagnosis to 277 clinical vignettes using multiple preoperative laboratory criteria for PJI. The undecided rate, interobserver agreement, and accuracy of physicians were characterized relative to the 2013 Musculoskeletal Infection Society (MSIS) gold standard and compared to the accuracy of each individual laboratory test for PJI. Results Physicians interpreting multiple criteria for PJI demonstrated high undecided diagnosis rates (mean=23.5%), poor interobserver agreement (kappa range=0.49-0.63), and mean accuracy of 90.8% (range:85.8%-97.4%) compared to the 2013 MSIS gold standard. The group of academic arthroplasty surgeons had a lower rate of undecided diagnoses than community arthroplasty surgeons (16.2% vs. 29.1%; p<0.0001) or ID specialists (16.2% vs. 25.1%; p<0.0001). Academic arthroplasty surgeons also exhibited a higher interobserver agreement than community arthroplasty surgeons (kappa = 0.63 (95%CI:0.59-0.68) vs. 0.49 (95%CI:0.44-0.54)). Mean physician accuracy (90.8%) was inferior to the alpha-defensin laboratory test (96.0%;p=0.0034) and the alpha-defensin lateral-flow test (94.6%;p=0.036), comparable to synovial fluid white blood cells (SF-WBC) (93.3%;p=0.17) and synovial fluid polymorphonuclear cell % (SF-PMN%) (94.0%;p=0.11), and superior to the erythrocyte sedimentation rate (ESR) (86.2%;p<0.0001) and C-reactive protein (CRP) (84.6%;p<0.0001). Only two academic arthroplasty surgeons in this study were able to outperform every individual test for PJI by combining multiple criteria to make a diagnosis. Conclusion Although multiple-criterion scoring systems may outperform individual tests for diagnosing PJI in the research setting, it appears that the complexity of using multiple tests to diagnose PJI causes indecision and variability among physicians. Physician use of multiple preoperative criteria to diagnose PJI is less accurate than the strict algorithmic calculation of the diagnosis as achieved in research. In fact, most physicians in this study would have improved their diagnostic accuracy for PJI by simply utilizing a single good test to make the diagnosis, instead of trying to combine multiple tests into a decision. We propose that less complex diagnostic criteria should be explored for routine clinical utilization.

Interobserver and Intra-Observer Reliability of a New Prognostic Classification for Foot and Ankle Diabetic Charcot Arthropathy

Category: Diabetes; Ankle; Hindfoot; Midfoot/Forefoot; Other Introduction/Purpose: Diabetic Charcot arthropathy of foot and ankle is a devastating, progressive destruction of bone and joint integrity affecting one or more joints. Although several classification systems exist for Charcot arthropathy of foot and ankle depending on developmental stages or anatomic regions affected, the majority of these classifications are mainly descriptive, and none of these classifications specified the management or the outcome with relation to the classification stage. Recently, a new prognostic classification (Mansoura Classification) was published, however the reliability of such classification is still unknown. The aim of this study was to evaluate both interobserver and intra-observer reliability of Mansoura classification for foot and ankle Charcot arthropathy. Methods: Mansoura classification for foot and ankle Charcot arthropathy was presented to participants at an international foot and ankle course, then participants were given colored printed copy of the classification table (table 1) and diagrams. Thereafter, twenty cases of foot and ankle Charcot were presented to participants and they were asked to rate each case according to the classification. The level of experience of all the participants was collected and classified as less than 5 years, 5-10 years, 10-15 years, 15-20 years and more than 20 years. Furthermore, six raters who were specialized in foot and ankle surgery were asked to repeat the classification of the cases after three weeks from the initial evaluation. Kappa statistics was used to evaluate both interobserver and intra-observer reliability using STATA 14.2 program, and results were interpreted as 0.00-0.20 Slight, 0.21-0.40 Fair, 0.41-0.60 Moderate, 0.61-0.80 Substantial; and 0.81-1.00 Excellent. Results: Sixty-one participants with different level of experience completed the evaluation of the twenty cases of foot and ankle Charcot. The interobserver reliability for all participants was moderate (Kappa=0.5). Further analysis according to the level of experience based on the years of practice of orthopedic surgery showed similar moderate reliability (Kappa range 0.4 to 0.57). Intra-observer reliability was excellent for all six participants, Kappa range 0.81 to 0.93. Further analysis of the interobserver reliability of the former six raters who were specialized in foot and ankle surgery showed substantial reliability; Kappa=0.67. Conclusion: Mansoura classification for Charcot arthropathy of foot and ankle has an excellent intra-observer reliability. Although the overall interobserver reliability among orthopedic surgeons was moderate and comparable to other classifications, the interobserver reliability for orthopedic surgeons who were specialized in foot and ankle surgery was substantial. Therefore, Mansoura classification for foot and ankle Charcot has an acceptable reliability and could be promising in the evaluation and guiding the management of such cases.

Development and validation of a machine-learning algorithm to predict the relevance of scientific articles within the field of teratology

The Dutch Teratology Information Service Lareb counsels healthcare professionals and patients about medication use during pregnancy and lactation. To keep the evidence up to date, employees perform a standardized weekly PubMed query where relevant literature is identified manually. We aimed to develop an accurate machine-learning algorithm to predict the relevance of PubMed entries, thereby reducing the labor-intensive task of manually screening the articles. We fine-tuned a pre-trained natural language processing transformer model to identify relevant entries. We split 15,540 labeled entries into case-control-balanced train, validation, and test datasets. Additionally, we externally validated the model prospectively with 1288 labeled entries obtained from weekly queries after developing the model. This dataset was also independently labeled by a team of six experienced human raters to evaluate our model’s performance. The validation of our machine learning model on the retrospectively collected outheld dataset obtained an area under the sensitivity-versus-specificity curve of 89.3 % (CI: 88.2– 90.4). In the prospective external validation of the model, our model classified relevant literature with a sensitivity versus specificity curve area of 87.4 % (CI: 85.0–89.8). Our model achieved a higher sensitivity than the human raters’ team without sacrificing too much specificity. The team of human raters showed weak to moderate levels of agreement in their article classifications (kappa range 0.40–0.64). The human selection of the latest relevant literature is indispensable to keep the teratology information up to date. We show that automatic preselection of relevant abstracts using machine learning is possible without sacrificing the selection performance.

Reliability and validity of 2-dimensional video analysis for a running task: A systematic review

ObjectiveEvaluate the reliability and validity of 2-dimensional (2D) video-based motion analysis during running. MethodsA systematic search of MEDLINE, Cochrane Library, EMBASE, CINAHL, PEDro, SPORTDiscus, and IEEE Xplore was conducted in March 2020 and updated in May 2021. We included studies assessing reliability and/or validity of 2D video-based motion analysis (gold standard: 3D motion analysis) during running. Results11 studies (251 runners; mean age range: 18.7–37.0 years; 57.4% female; 63.7% injury-free) met inclusion criteria. Eight studies examined kinematics of the pelvis/hip, eight of the knee, and six of the ankle/foot. Low-to-moderate risk of bias was present in all studies. Heterogeneous study designs, measurement methods, and statistical approaches across studies precluded statistical synthesis. Intrarater reliability [Interclass correlation coefficient (ICC) range: 0.56–1.00; kappa range: 0.49–0.81] was better than interrater reliability (ICC range: 0.31–1.00; kappa range 0.00–0.85). ICC values for validity were poor to good (0.06–0.89). One study examining foot strike pattern found good to excellent validity (using Gwet AC statistics) when movement kinematics were categorized. ConclusionsA wide range of methods were reported in 2D video-based motion analysis of joint angular kinematics during a running task. Further research to develop standardized 2D video-based motion analysis for running is needed. Categorizing movement patterns may be more useful than angularly quantifying joint kinematics.

How Reliable Are Gene Expression-Based and Immunohistochemical Biomarkers Assessed on a Core-Needle Biopsy? A Study of Paired Core-Needle Biopsies and Surgical Specimens in Early Breast Cancer.

Simple SummaryThis study aims to assess the reliability of using a core-needle biopsy (CNB) for preoperative tumor characterization using gene expression analysis and conventional immunohistochemistry (IHC) analysis for clinical biomarkers in early breast cancer. Obtaining a preoperative CNB is a standard procedure in the evaluation of a primary breast cancer, with previous studies suggesting that it can be considered trustworthy to perform immunohistochemical analysis on CNB samples. However, little research has been carried out to evaluate whether gene expression-based biomarker assessment can be carried out reliably on the preoperative CNB. This information is important as genomic profiling is gaining ever-increasing importance in the treatment of early breast cancer.In early breast cancer, a preoperative core-needle biopsy (CNB) is vital to confirm the malignancy of suspected lesions and for assessing the expression of treatment predictive and prognostic biomarkers in the tumor to choose the optimal treatments, emphasizing the importance of obtaining reliable results when biomarker status is assessed on a CNB specimen. This study aims to determine the concordance between biomarker status assessed as part of clinical workup on a CNB compared to a medically untreated surgical specimen. Paired CNB and surgical specimens from 259 patients that were part of the SCAN-B cohort were studied. The concordance between immunohistochemical (IHC) and gene expression (GEX) based biomarker status was investigated. Biomarkers of interest included estrogen receptor (ER; specifically, the alpha variant), progesterone receptor (PgR), Ki67, HER2, and tumor molecular subtype. In general, moderate to very good correlation in biomarker status between the paired CNB and surgical specimens was observed for both IHC assessment (83–99% agreement, kappa range 0.474–0.917) and GEX assessment (70–97% agreement, kappa range 0.552–0.800), respectively. However, using IHC, 52% of cases with low Ki67 status in the CNB shifted to high Ki67 status in the surgical specimen (McNemar’s p = 0.011). Similarly, when using GEX, a significant shift from negative to positive ER (47%) and from low to high Ki67 (16%) was observed between the CNB and surgical specimen (McNemar’s p = 0.027 and p = 0.002 respectively). When comparing biomarker status between different techniques (IHC vs. GEX) performed on either CNBs or surgical specimens, the agreement in ER, PgR, and HER2 status was generally over 80% in both CNBs and surgical specimens (kappa range 0.395–0.708), but Ki67 and tumor molecular subtype showed lower concordance levels between IHC and GEX (48–62% agreement, kappa range 0.152–0.398). These results suggest that both the techniques used for collecting tissue samples and analyzing biomarker status have the potential to affect the results of biomarker assessment, potentially also impacting treatment decisions and patient survival outcomes.

Psychiatric diagnostic dilemmas among people with intellectual and developmental disabilities.

Research regarding the accuracy of co-morbid psychiatric diagnoses in individuals with intellectual and developmental disabilities (IDD) is sparse. Yet correct diagnostic assignment is vital so that effective and appropriate treatment can be implemented, especially for the large numbers of individuals requiring expensive and restrictive behavioural health crisis services. A retrospective review of de-identified data from multidisciplinary specialty team assessments completed for 50 individuals with ID (IntellectualDisability) with and without ASD and unresolved behavioural health challenges was conducted. The accuracy and reliability of the psychiatric diagnoses upon referral were compared with the diagnoses after the comprehensive team evaluation, and within-individual diagnostic agreement was calculated. The agreement between the Mood and Anxiety Semi-Structured interview tool (MASS) and the full team evaluation was also calculated. The influence of demographic and clinical characteristics on diagnostic agreement was explored. The most common chief complaints upon referral were aggression to others and self-injurious behaviour. Individuals were taking a median of six medications (interquartile range: 5 to 7); 80% were taking an antipsychotic medication. The most common medical conditions were constipation (70%) and gastroesophageal reflux disease (52%). Measures of interrater reliability of the referral diagnoses with the team assessment were below 0.5 (kappa range: -0.04 to 0.39), with the exception of ruling out dementia (kappa=0.85). The interrater reliability estimates for the MASS evaluations for depression and anxiety were higher (kappa=0.69 and 0.64) and reflected higher sensitivity and PPV. The odds of any referral diagnosis being confirmed by team evaluation were low: 0.25 (range: 0 to 0.67). The level of diagnostic agreement for each patient was not significantly attributable to demographic or clinical characteristics, although effect sizes indicate a possible positive relationship to age and the number of prescribed psychotropic medications at referral. Individuals in the current study had serious psychiatric and behavioural problems despite psychiatric care in their communities. The majority of psychiatric diagnoses provided upon referral were not supported by the multidisciplinary specialty team's assessment. In addition to possible diagnostic inaccuracy, the group in the study suffered from multiple medical co-morbidities and were exposed to polypharmacy. Results emphasise the importance of multidisciplinary evaluation by clinicians with expertise in neurodevelopmental disabilities when people with ID with and without ASD have complex behavioural health needs that are unresponsive to usual care. In addition, based on agreement with the full team evaluation, the MASS shows promise as an assessment tool, especially with regards to identifying anxiety and depression.

Treatment dropout among veterans and their families: Quantitative and qualitative findings.

Psychotherapy noncompletion rates for veterans and their families are high. This study sought to (a) measure noncompletion rates of such patients at a university-based treatment center, (b) compare veteran and family member attrition rates, (c) identify dropout predictors, and (d) explore clinicians' perspectives on treatment noncompletion. Using quantitative and qualitative approaches, we analyzed demographic and clinical characteristics of 141 patients (90 military veterans; 51 family members) in a university treatment center. We defined dropout as not completing the time-limited therapy contract. Reviewing semistructured interview data assessing clinicians' perspectives on their patients' dropout, three independent raters agreed on key themes, with interrater coefficient kappa range .74 to 1. Patient attrition was 24%, not differing significantly between veterans and family members. Diagnosis of major depression (MDD) and exposure-based therapies predicted noncompletion, as did higher baseline Hamilton Depression Rating Scale (HDRS) total scores, severe depression (HDRS > 20), lack of Beck Depression Inventory weekly improvement, and history of military sexual trauma. Clinicians mostly attributed noncompletion to patient difficulties coping with intense emotions, especially in exposure-based therapies. Noncompletion rate at this study appeared relatively low compared to other veteran-based treatment centers, if still unfortunately substantial. Patients with comorbid MDD/PTSD and exposure-based therapies carried greater noncompletion risk due to the MDD component, and this should be considered in treatment planning. Ongoing discussion of dissatisfaction and patient discontinuation, in the context of a strong therapeutic alliance, might reduce noncompletion in this at-risk population. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Using artificial intelligence to support rapid, mixed-methods analysis: Developing an automated qualitative assistant (AQUA).

Context: Qualitative research - crucial for understanding human behavior - remains underutilized, in part due to the time and cost of annotating qualitative data (coding). Artificial intelligence (AI) has been suggested as a means to reduce those burdens. Older AI techniques (Latent Semantic Indexing / Latent Dirichlet Allocation (LSI/LDA)) have fallen short, in part because qualitative data is rife with idiom, non-standard expressions, and jargon. Objective: To develop an AI platform using updated techniques to augment qualitative data coding. Study Design and Analysis: We previously completed traditional qualitative analysis of a large dataset, with 11 qualitative categories and 72 subcategories (categories), and a final Cohen's kappa ≥ 0.65 as a measure of human inter-coder reliability (ICR) after coding. We built our Automated Qualitative Assistant (AQUA) using a semi-classical approach, replacing LSI/LDA with a graph-theoretic topic extraction and clustering method. AQUA was given the previously-identified qualitative categories and tasked with coding free-text data into those categories. Item coding was scored using cosine-similarity. Population Studied: Pennsylvanian adults. Instrument: Free-text responses to five open ended questions related to the COVID-19 pandemic (e.g. "What worries you most about the COVID-19 pandemic?"). Outcome Measures: AQUA's coding was compared to human coding using Cohen's kappa. This was done on all categories in aggregate, and also on category clusters to identify category groups amenable to AQUA support. AQUA's time to complete coding was compared to the time taken by the human coding team. Dataset: Five unlimited free-text survey answers from 538 responders. Results: AQUA's kappa for all categories was low (kappa~0.45), reflecting the challenge of automated analysis of diverse language. However, for several 3-category combinations (with less linguistic diversity), AQUA performed comparably to human coders, with an ICR kappa range of 0.62 to 0.72 based on test-train split. AQUA's analysis (including human interpretation) took approximately 5 hours, compared to approximately 30 person hours for traditional coding. Conclusions: AQUA enables qualitative researchers to identify categories amenable to automated coding, and to rapidly conduct that coding on the entirety of very large datasets. This saves time and money, and avoids limitations inherent in limiting qualitative analysis to limited samples of a given dataset.