Intraclass Correlation Coefficient Statistics Research Articles

Intra-rater and inter-rater reliability of 3D facial measurements

Three-dimensional (3D) body scanning technology has applications for obtaining anthropometric data in human-centered and product development fields. The reliability of 3D measurements gathered from 3D scans must be assessed to understand the degree to which this technology is appropriate for use in place of manual anthropometric methods. The intra- and inter-rater reliabilities of 3D facial measurements were assessed among four novice raters using 3D landmarking. Intraclass correlation coefficient (ICC) statistics were calculated for the 3D measurement data collected in three phases to assess baseline reliabilities and improvements in reliabilities as the result of additional training and experience. Based on the results of this study, the researchers found that the collection of 3D measurement data, by multiple raters and using 3D landmarking methods, yielded a high percentage of ICC statistics in the good to excellent (>0.75 ICC) reliability range. Rater training and experience were important considerations in improving intra- and inter-rater reliabilities.

A meta-analysis of microbial thermal inactivation in low moisture foods

Microbial thermal inactivation in low moisture foods is challenging due to enhanced thermal resistance of microbes and low thermal conductivity of food matrices. In this study, we leveraged the body of previous work on this topic to model key experimental features that determine microbial thermal inactivation in low moisture foods. We identified 27 studies which contained 782 mean D-values and developed linear mixed-effect models to assess the effect of microorganism type, matrix structure and composition, water activity, temperature, and inoculation and recovery methods on cell death kinetics. Intraclass correlation statistics (I2) and conditional R2 values of the linear mixed effects models were: E. coli (R2-0.91, I2-83%), fungi (R2-0.88, I2-85%), L. monocytogenes (R2-0.84, I2-75%), Salmonella (R2-0.69, I2-46%). Finally, global response surface models (RSM) were developed to further study the non-linear effect of aw and temperature on inactivation. The fit of these models varied by organisms from R2 0.88 (E. coli) to 0.35 (fungi). Further dividing the Salmonella data into individual RSM models based on matrix structure improved model fit to R2 0.90 (paste-like products) and 0.48 (powder-like products). This indicates a negative relationship between data diversity and model performance.

Factors Associated With Decision to Use and Dosing of Sugammadex in Children: A Retrospective Cross-Sectional Observational Study.

Sugammadex was initially approved for reversal of neuromuscular blockade in adults in the United States in 2015. Limited data suggest sugammadex is widely used in pediatric anesthesia practice however the factors influencing use are not known. We explore patient, surgical, and institutional factors associated with the decision to use sugammadex versus neostigmine or no reversal, and the decision to use 2 mg/kg vs 4 mg/kg dosing. Using data from the Multicenter Perioperative Outcomes Group (MPOG) database, an EHR-derived registry, we conducted a retrospective cross-sectional study. Eligible cases were performed between January 1, 2016 and December 31, 2020, for children 0 to 17 years at US hospitals. Cases involved general anesthesia with endotracheal intubation and administration of rocuronium or vecuronium. Using generalized linear mixed models with institution and anesthesiologist-specific random intercepts, we measured the importance of a variety of patient, clinician, institution, anesthetic, and surgical risk factors in the decision to use sugammadex versus neostigmine, and the decision to use a 2 mg/kg vs 4 mg/kg dose. We then used intraclass correlation statistics to evaluate the proportion of variance contributed by institution and anesthesiologist specifically. There were 97,654 eligible anesthetics across 30 institutions. Of these 47.1% received sugammadex, 43.1% received neostigmine, and 9.8% received no reversal agent. Variability in the choice to use sugammadex was attributable primarily to institution (40.4%) and attending anesthesiologist (27.1%). Factors associated with sugammadex use (compared to neostigmine) include time from first institutional use of sugammadex (odds ratio [OR], 1.08, 95% confidence interval [CI], 1.08-1.09, per month, P < .001), younger patient age groups (0-27 days OR, 2.59 [2.00-3.34], P < .001; 28 days-1 year OR, 2.72 [2.16-3.43], P < .001 vs 12-17 years), increased American Society of Anesthesiologists [ASA] physical status (ASA III: OR, 1.32 [1.23-1.42], P < .001 ASA IV OR, 1.71 [1.46-2.00], P < .001 vs ASA I), neuromuscular disease (OR, 1.14 (1.04-1.26], P = .006), cardiac surgery (OR, 1.76 [1.40-2.22], P < .001), dose of neuromuscular blockade within the hour before reversal (>2 ED95s/kg OR, 4.58 (4.14-5.07], P < .001 vs none), and shorter case duration (case duration <60 minutes OR, 2.06 [1.75-2.43], P < .001 vs >300 minutes). Variation in sugammadex use was primarily explained by institution and attending anesthesiologist. Patient factors associated with the decision to use sugammadex included younger age, higher doses of neuromuscular blocking agents, and increased medical complexity.

Accuracy and replicability of identifying eyelid tremor as an indicator of recent cannabis smoking

Introduction Cannabis intoxication may increase the risk of motor vehicle crashes. However, reliable methods of assessing cannabis intoxication are limited. The presence of eyelid tremors is among the signs of cannabis use identified under the Drug Evaluation and Classification Program of the International Association of Chiefs of Police. Our objectives were to assess the accuracy and replicability of identifying eyelid tremor as an indicator of recent cannabis smoking using a blinded, controlled study design. Methods Adult subjects (N = 103) were recruited into three groups based on their cannabis use history: daily, occasional, and no current cannabis use. Participants’ closed eyelids were video recorded for 30 seconds by infrared videography goggles before and at a mean ± standard deviation time of 71.4 ± 4.6 minutes after the onset of a 15-minute interval of ad libitum cannabis flower smoking or vaping. Three observers with expertise in neuro-ophthalmology and medical toxicology were trained on exemplar videos of eyelids to reach a consensus on how to grade eyelid tremor. Without knowledge of subjects’ cannabis use history or time point (pre- or post-smoking), observers reviewed each video for eyelid tremor graded as absent, slight, moderate, or severe. During subsequent data analysis, this score was further dichotomized as a consensus score of absent (absent/slight) or present (moderate/severe). Results Kappa and intraclass correlation coefficient statistics demonstrated moderate agreement among the coders, which ranged from 0.44–0.45 and 0.58–0.61, respectively. There was no significant association between recent cannabis use and the observers’ consensus assessment that eyelid tremor was present, and cannabis users were less likely to have tremors (odds ratio: 0.75; 95 percent confidence interval: 0.25, 2.40). The assessment of eyelid tremor as an indicator of recent cannabis smoking had a sensitivity of 0.86, specificity of 0.18, and accuracy of 0.64. Discussion Eyelid tremor has fair sensitivity but poor specificity and accuracy for identification of recent cannabis use. Inter-rater reliability for assessment of eyelid tremor was moderate for the presence and degree of tremor. The weak association between recent cannabis use and eyelid tremor does not support its utility in identifying recent cannabis use. Limitations Videos were recorded at only one time point after cannabis use. Adherence to abstinence could not be strictly supervised. Due to regulatory restrictions, we were unable to control the cannabis product used or administer a fixed Δ9-tetrahydrocannabinol dose. Participants were predominately non-Hispanic and White. Conclusions In a cohort of participants with a range of cannabis use histories, acute cannabis smoking was not associated with the presence of eyelid tremor, regardless of cannabis use history, at 70 minutes post-smoking. Additional research is needed to identify the presence of eyelid tremor accurately, determine the relationship between cannabis dose and timeline in relation to last cannabis use to eyelid tremor, and determine how it should be, if at all, utilized for cannabis Drug Recognition Evaluator examinations.

Analytical Rubric Development Study for A2 Level Speaking Skills in Teaching Turkish as a Foreign Language

Speaking skill is one of the most important skills in learning a foreign language and realizing the act of communication. The more the student improves himself in speaking skills, the more self-confidence he will increase. In the development of speaking skill, measurement and evaluation of the skill is as important as all the activities carried out during the lesson. It is important for the student to receive feedback on his speaking skill and to know how scoring is done. In this research, it is aimed to develop a rubric that will guide both students and teachers in order to evaluate the speaking performance of students at the A2 level of teaching Turkish as a foreign language. In the study, the validity and reliability studies of the rubric were carried out. A group of four field experts was formed in the study. As a result of the scores made by four field experts, the reliability between the raters was calculated according to the Intraclass Correlation statistics in the SPSS 25 Program. According to the feedback from the experts, it has been revealed that it is a tool that both teachers and students can use as an assessment tool. It is thought that the rubric will contribute to the researchers while creating a tool at the same level or at other levels, and will guide the teachers and learners in the stages of measuring and evaluating the A2 level speaking skill of teaching Turkish as a foreign language.

Integrating pragmatic and implementation science randomized clinical trial approaches: a PRagmatic Explanatory Continuum Indicator Summary-2 (PRECIS-2) analysis

BackgroundOver the past two decades, pragmatic and implementation science clinical trial research methods have advanced substantially. Pragmatic and implementation studies have natural areas of overlap, particularly relating to the goal of using clinical trial data to leverage health care system policy changes. Few investigations have addressed pragmatic and implementation science randomized trial methods development while also considering policy impact.MethodsThe investigation used the PRagmatic Explanatory Continuum Indicator Summary-2 (PRECIS-2) and PRECIS-2-Provider Strategies (PRECIS-2-PS) tools to evaluate the design of two multisite randomized clinical trials that targeted patient-level effectiveness outcomes, provider-level practice changes and health care system policy. Seven raters received PRECIS-2 training and applied the tools in the coding of the two trials. Descriptive statistics were produced for both trials, and PRECIS-2 wheel diagrams were constructed. Interrater agreement was assessed with the Intraclass Correlation (ICC) and Kappa statistics. The Rapid Assessment Procedure Informed Clinical Ethnography (RAPICE) qualitative approach was applied to understanding integrative themes derived from the PRECIS-2 ratings and an end-of-study policy summit.ResultsThe ICCs for the composite ratings across the patient and provider-focused PRECIS-2 domains ranged from 0.77 to 0.87, and the Kappa values ranged from 0.25 to 0.37, reflecting overall fair-to-good interrater agreement for both trials. All four PRECIS-2 wheels were rated more pragmatic than explanatory, with composite mean and median scores ≥ 4. Across trials, the primary intent-to-treat analysis domain was consistently rated most pragmatic (mean = 5.0, SD = 0), while the follow-up/data collection domain was rated most explanatory (mean range = 3.14–3.43, SD range = 0.49–0.69). RAPICE field notes identified themes related to potential PRECIS-2 training improvements, as well as policy themes related to using trial data to inform US trauma care system practice change; the policy themes were not captured by the PRECIS-2 ratings.ConclusionsThe investigation documents that the PRECIS-2 and PRECIS-2-PS can be simultaneously used to feasibly and reliably characterize clinical trials with patient and provider-level targets. The integration of pragmatic and implementation science clinical trial research methods can be furthered by using common metrics such as the PRECIS-2 and PRECIS-2-PS. Future study could focus on clinical trial policy research methods development.Trial registrationDO-SBIS ClinicalTrials.gov NCT00607620. registered on January 29, 2008. TSOS ClinicalTrials.gov NCT02655354, registered on July 27, 2015.

Mean kurtosis-Curve (MK-Curve) correction improves the test-retest reproducibility of diffusion kurtosis imaging at 3 T.

Diffusion kurtosis imaging (DKI) is applied to gain insights into the microstructural organization of brain tissues. However, the reproducibility of DKI outside brain white matter, particularly in combination with advanced estimation to remedy its noise sensitivity, remains poorly characterized. Therefore, in this study, we investigated the variability and reliability of DKI metrics while correcting implausible values with a fit method called mean kurtosis (MK)-Curve. A total of 10 volunteers (four women; age: 41.4 ± 9.6 years) were included and underwent two MRI examinations of the brain. The images were acquired on a clinical 3-T scanner and included a T1-weighted image and a diffusion sequence with multiple diffusion weightings suitable for DKI. Region of interest analysis of common kurtosis and tensor metrics derived with the MK-Curve DKI fit was performed, including intraclass correlation (ICC) and Bland-Altman (BA) plot statistics. A p value of less than 0.05 was considered statistically significant. The analyses showed good to excellent agreement of both kurtosis tensor- and diffusion tensor-derived MK-Curve-corrected metrics (ICC values: 0.77-0.98 and 0.87-0.98, respectively), with the exception of two DKI-derived metrics (axial kurtosis in the cortex: ICC = 0.68, and radial kurtosis in deep gray matter: ICC = 0.544). Non-MK-Curve-corrected kurtosis tensor-derived metrics ranged from 0.01 to 0.52 and diffusion tensor-derived metrics from 0.06 to 0.66, indicating poor to moderate reliability. No structural bias was observed in the BA plots for any of the diffusion metrics. In conclusion, MK-Curve-corrected DKI metrics of the human brain can be reliably acquired in white and gray matter at 3 T and DKI metrics have good to excellent agreement in a test-retest setting.

Development and validation of a novel cosmetics safety assessment scale (CSAS): Factual understanding of cosmetic safety and fostering international awareness

Falsified cosmetics are increasingly common especially online through social media networks and mobile applications. This study developed and validated a novel tool to evaluate the safety of cosmetics and personal care products in the United Araba Emirates (UAE). This is methodological validation study and the data were derived from a cross-sectional study conducted on students and staff at Ajman University (AU) in the UAE. The study sample was selected via simple random sampling. The link to the survey was sent to potential respondents via email, and the responses were analysed using SPSS version 26. Content validity, factor analysis, and known group validity were employed to construct and validate an instrument that will enable the identification of cosmetics safety. The instrument's reliability was evaluated using test-retest reliability, internal consistency, item internal consistency (IIC), and intraclass correlation coefficients (ICCs). The study sample included 978 participants. The content validity index for the final 24-item scale was 0.84. The Kaiser-Meyer-Olkin value was 0.959 with a statistically significant Bartlett's test of sphericity (p &lt;0.001). Factor analysis presented a three-component model. PCFA analysis found good fit values with 0.960 for the normed fit index, 0.977 for the comparative fit index, and 0.987 for the Tucker Lewis Index. All values were in excess of 0.95, and the root mean square error of approximation was below 0.06 (0.03); thus, the model had a good fit. Cronbach's alpha also showed good consistency of the overall instrument (0.963), and all factors had a Cronbach's alpha above 0.70. Each item on the instrument met the IIC correlation standard of ≥ 0.40, and there were good overall ICC statistics of 0.963 (0.959-0.966) for the instrument as a whole with statistical significance (p &lt; 0.001). The instrument's test-retest reliability was assessed by correlating the respondents' identification scores at two time points with a four-week gap revealing a correlation coefficient of 0.870 (p-value &lt;0.01). Participants holding a bachelor's degree were more likely to be able to identify safe and authentic cosmetics than those with a high-school educational level (p = 0.015). This study developed a novel validated instrument to determine the safety of cosmetics. The final questionnaire uses 24 items on three dimensions (13 items on hazard information, eight items on product identity, and three items on product handling and storage). The tool is concise and easy to complete, and it is suitable for use among the general population. The use of this instrument can promote greater collaboration between the consumer health regulatory authorities and inspection authorities thus increasing consumer satisfaction and public participation.

Clinical validation of a smartphone application for automated wound measurement in patients with venous leg ulcers.

Chronic wounds are associated with significant clinical, economic and quality-of-life burden. Despite the variety of wound imaging systems available in the market for wound assessment and surveillance, few are clinically validated among patients of Asian ethnicity. We aimed to clinically validate the accuracy of a smartphone wound application (Tissue Analytics [TA], Net Health Systems Inc, Florida, USA), versus conventional wound measurements (visual approximation and paper rulers), in patients of Asian ethnicity with venous leg ulcers (VLU). A prospective cohort study of patients presenting with VLU to a specialist wound nurse clinic over a 5-week duration was conducted. Each patient received seven wound measurements: one by a trained wound nurse clinician, and three separate wound measurements using TA on each of the iOS and Android operating systems. Inter-rater and intra-rater reliability between clinical and TA-based measurements were analysed using intra-class correlation statistics, with values of <0.5, 0.5 to 0.75, 0.75 to 0.9, and >0.9 indicating poor, moderate, good and excellent reliability, respectively. 82 patients (51% males), with a mean age at 65.8 years, completed the 5-week study duration. 25 (30%) had underlying diabetes mellitus. Chinese, Malay and Indian ethnicity comprised 68%, 12% and 11%, respectively. The VLU healed in 26 (32%) of patients within the study period. In total, 358 wound episodes with 2334 wound images were analysed. Inter-rater reliability for length, width and area between wound nurse measurements and TA application measurements was good (range 0.799-0.919, P < 0.001). Separate measurements of intra-rater reliability for length, width and area within the iOS or Android systems were excellent (range 0.967-0.985 and range 0.977-0.984 respectively, P < 0.001). Inter-rater reliability between TA used on the iOS and Android systems was also excellent (0.987-0.989, P < 0.001). Tissue Analytics, a smartphone wound application, is a useful adjunct for wound assessment and surveillance in VLU patients of Asian ethnicity.

Development and Validation of a Tool to Improve Community Pharmacists' Surveillance Role in the Safe Dispensing of Herbal Supplements.

Background: There has been an appreciable increase in the use of herbal supplements, including immune boosters, during the current COVID-19 pandemic. However, there are concerns with falsified herbal supplements. Objectives: Developed a new questionnaire that can potentially help community pharmacists identify the extent of falsified herbal supplements. Methods: A 9-month cross sectional study was conducted among 500 community pharmacies across United Arab Emirates. A new 5-factor, 24-itmes scale was developed based on current labelling requirements across countries and piloted. This included seven items on identified uses and contraindication, seven items on hazard identification, four items on product identity, three items on packaging and product insert and three items on product handling and storage. The face and content validity of the scale was assessed via the content validity index (CVI). Its construct validity was tested using an exploratory factor analysis (EFA) via principally component analysis (PCA). The model was subsequently confirmed through partial confirmatory factor analysis (PCFA). Its reliability was assessed via test-retest reliability, internal consistency, item internal consistency (IIC), and intraclass correlation coefficients (ICCs). Results: The CVI of the finalized questionnaire was 0.843. The Kaiser-Meyer-Olkin measure of sampling adequacy was 0.891, and Bartlett’s test of sphericity indicated significance (p-value < 0.001). Confirmation of the subsequent 5-domains was achieved through PCFA using maximum likelihood analysis with oblimin rotation. The PCFA obtained values was 0.962 for NFI, 0.977 for CFI, and 0.987 for the Tucker Lewis Index. All values were greater than 0.95, and the RMSEA value was 0.03 (i.e., less than 0.06). Consequently, the model had a good fit. All domains demonstrated Cronbach’s alpha coefficients above 0.70, with 0.940 for the full instrument. Meanwhile, all items met the IIC correlation standard of ≥0.40. The instrument presented good ICC statistics of 0.940 (0.928–0.950) as well as statistical significance (p < 0.001). Community pharmacists with more than 10 experience years were more likely to identify falsified herbal supplements compared to those with 1–10 years experience (p < 0.001). Conclusion: This study developed and validated a new instrument to identify safe herbal supplements, which should enhance the role of the community pharmacists in the safe and effective treatment of suitable patients with herbal supplements.

Asymptomatic sacroiliitis detected by magnetic resonance enterography in patients with Crohn's disease: prevalence, association with clinical data, and reliability among radiologists in a multicenter study of adult and pediatric population

Patients with Crohn's disease (CD) usually undergo magnetic resonance enterography (MRE) for evaluating small bowel involvement. Musculoskeletal symptoms are the most frequent extraintestinal manifestation in inflammatory bowel diseases, especially in CD, with sacroiliitis at imaging occurring in about 6-46% of patients and possibly correlating with axial spondyloarthritis. The primary study aim was to assess the prevalence of sacroiliitis in adult and pediatric patients with CD performing an MRE. We also evaluated the inter-rater agreement for MRE sacroiliitis and the association between sacroiliitis and patients' clinical data. We retrospectively identified 100 adult and 30 pediatric patients diagnosed with CD who performed an MRE between December 2012 and May 2020 in three inflammatory bowel disease centers. Two radiologists assessed the prevalence of sacroiliitis at MRE. We evaluated the inter-rater agreement for sacroiliitis with Cohen's kappa and intraclass correlation coefficient statistics and assessed the correlation between sacroiliitis and demographic, clinical, and endoscopic data (Chi-square and Fisher's tests). The prevalence of sacroiliitis at MRE was 20% in adults and 6.7% in pediatric patients. The inter-rater agreement for sacroiliitis was substantial (k = 0.62, p < 0.001) in the adults and moderate (k = 0.46, p = 0.011) in the pediatric cohort. Age ≥ 50years and the time between CD diagnosis and MRE (≥ 86.5months) were significantly associated with sacroiliitis in adult patients (p = 0.049 and p = 0.038, respectively). Sacroiliitis is a frequent and reliable abnormality at MRE in adult patients with CD, associated with the age of the patients ≥ 50years and CD duration.

Abstract P1-08-10: Ki67 assessment based on international Ki67 working group recommendations and correlation with 21-gene assay results in a large integrated health care system: We might not be there yet

Abstract Background: While substantial evidence indicates that Ki67, a marker of proliferation, is strongly associated with breast cancer outcomes, its clinical utility has been limited given concerns about scoring inter-rater reliability and appropriate cut points. Nonetheless, Ki67 has been used in multiple clinical trials and results from POETIC indicated that low baseline Ki67 (ie, &amp;lt;10%) predicts good prognosis in postmenopausal women with hormone sensitive, early breast cancer. Further, the International Ki67 Working Group (IKWG) has developed website-based training materials to improve reproducibility of Ki67 scoring by immunohistochemistry (IHC) and recently considered the marker to be sufficiently validated to support treatment decisions in early ER+ breast cancer (≤ 5 no chemotherapy, ≥30 chemotherapy indicated). Our aims were to examine Ki67 scoring reproducibility following IKWG training and the extent to which low or high scores could accurately identify patients with low or high 21-gene assay Recurrence Scores (RS) who could selectively avoid this test. Methods: Setting and study population. The study was conducted within Kaiser Permanente Northern California (KPNC), an integrated health care system with over 4.4 million enrollees. We included a random sample of women aged 50+ years at diagnosis of node-negative, ER+PR+HER2- breast cancer with the 21-gene assay done on their surgical specimen from 2018-2020 (n=307). Ki67 staining, training and scoring. We retrieved archived core biopsy specimens, which were sent to NeoGenomics Laboratories, Inc for Ki67 staining and scoring by image analysis (IA) using the hot spot counting method. In addition, two KPNC pathologists specializing in semiquantitative IHC scoring underwent IKWG training and independently scored all slides using the global counting method, blinded to each other and to readings by AI; weighted Ki67 scores were calculated. Analysis. We examined inter-rater reliability across pathologists using intraclass correlation (ICC) and Kappa statistics. We also examined the percent of patients with low Ki67 scores (≤ 5, &amp;lt;10) by each pathologist and by AI who also had low RS (&amp;lt;26) and the percent who had high Ki67 scores (≥30) who also had high RS (≥26). Results: Approximately 83% of patients were ages 50-69 years (median 63), 61% were non-Hispanic white and 93% were stage 1A. The ICC for Ki67 scores (log-transformed) by the two pathologists was 0.82; using a dichotomous cut point of &amp;lt;10 vs ≥10%, the Kappa for the inter-rater reliability was 0.65. Among patients with Ki67 scores of &amp;lt;10% by IA (n=81), pathologist 1 (n=120) or pathologist 2 (n=111), the percent with a RS of &amp;lt;26 was 95.1% for IA, 95.8% for pathologist 1, and 94.6% for pathologist 2. Among patients with Ki67 scores ≤5, the percentages were 90.9%, 92.6% and 91.8% for IA, pathologist 1 and pathologist 2, respectively. Among patients with Ki67 scores ≥30 by AI (n=69), pathologist 1 (n=41) or pathologist 2 (n=35), the percent who had a RS ≥26 was 27.5% for IA, 41.5% for pathologist 1 and 51.4% for pathologist 2. Results are improved if we exclude all 50 patients with weak PR by IHC (1-10%); for example, among patients with Ki67 scores of &amp;lt;10%, the percent with a RS of &amp;lt;26 was 97.1% for IA, 98.1% for pathologist 1, and 97.9% for pathologist 2. Conclusion: Among women aged 50+ years with node-negative, ER+PR+HER2- breast cancer in our setting, approximately 5-10% of patients with Ki67 scores of ≤ 5% or &amp;lt;10% on core biopsies would have high RS (≥26) on surgical specimens and over 48% of cases with Ki67 scores ≥30 would have low RS (&amp;lt;26), which may be insufficiently accurate for avoiding the 21-gene or other multi-gene assays. Future studies are needed to examine whether restricting Ki67 testing to ER+HER2- patients with PR &amp;gt;10% would improve its clinical validity. Citation Format: Veronica C. Shim, Robin J Baker, Wen Jing, Sally S Agersborg, Thomas K Lee, Wamda Goreal, Ninah Achacoso, Catherine Lee, Marvella Villasenor, Amy Lin, Malathy Kapali, Laurel A Habel. Ki67 assessment based on international Ki67 working group recommendations and correlation with 21-gene assay results in a large integrated health care system: We might not be there yet [abstract]. In: Proceedings of the 2021 San Antonio Breast Cancer Symposium; 2021 Dec 7-10; San Antonio, TX. Philadelphia (PA): AACR; Cancer Res 2022;82(4 Suppl):Abstract nr P1-08-10.

Empirical Determination of Transitory and Persistent Delinquency in Chilean Youth: Validation of the Criminal Engagement Severity Scale "EGED".

Evidence from several longitudinal studies has established the relevance of the approach of evolutionary criminology to understanding and intervening with adolescent criminal offenders, seeking to halt the criminal behavior before its potential consolidation in adulthood. The aim of this study is to present the psychometric properties of the Criminal Engagement Severity Scale (EGED) to discriminate between transitory and persistent delinquency in Chilean adolescents of both sexes. The characteristics of the sample are revealed through descriptive analyses, and evidence of validity and reliability is provided that show its discriminant capacity using ROC curves and odds ratios, measures of internal consistency (Cronbach’s α and McDonald’s Ω), intraclass correlation, and unidimensional statistics. The results indicate that the EGED adequately discriminates between transitory and persistent delinquency, so that its use in contexts of assessment and intervention with adolescent criminal offenders can be recommended, because it helps to determine the intensity of the intervention required.

MEG Node Degree Differences in Patients with Focal Epilepsy vs. Controls-Influence of Experimental Conditions.

Drug-resistant epilepsy can be most limiting for patients, and surgery represents a viable therapy option. With the growing research on the human connectome and the evidence of epilepsy being a network disorder, connectivity analysis may be able to contribute to our understanding of epilepsy and may be potentially developed into clinical applications. In this magnetoencephalographic study, we determined the whole-brain node degree of connectivity levels in patients and controls. Resting-state activity was measured at five frequency bands in 15 healthy controls and 15 patients with focal epilepsy of different etiologies. The whole-brain all-to-all imaginary part of coherence in source space was then calculated. Node degree was determined and parcellated and was used for further statistical evaluation. In comparison to controls, we found a significantly higher overall node degree in patients with lesional and non-lesional epilepsy. Furthermore, we examined the conditions of high/reduced vigilance and open/closed eyes in controls, to analyze whether patient node degree levels can be achieved. We evaluated intraclass-correlation statistics (ICC) to evaluate the reproducibility. Connectivity and specifically node degree analysis could present new tools for one of the most common neurological diseases, with potential applications in epilepsy diagnostics.

Interclinician agreement on the recognition of selected respiratory clinical signs in dogs and cats with abnormal breathing patterns

In humans, classification of abnormal breathing patterns (ABP) and recognition of ancillary respiratory signs are difficult, as reflected by poor-to-moderate interclinician agreement. The aims of this study were to assess interclinician agreement for respiratory sign recognition in dogs and cats and evaluate the influence of clinical experience on agreement. Dogs and cats with ABP were recruited from three hospitals. Included animals were evaluated by three clinicians at each hospital before therapeutic intervention. Consensual definitions for each respiratory clinical sign were provided to all clinicians. Interclinician agreement was measured via Fleiss’ kappa and intraclass correlation coefficient statistics. Influence of clinical experience on interobserver agreement was studied via mixed-effects logistic regression.One-hundred and fifteen dogs and 49 cats with ABP were recruited. Out of 12 clinical signs evaluated, only stertor (kappa, 0.80), stridor (kappa, 0.64), attenuation of heart/lung sounds (kappa, 0.60), and goose honking (kappa, 0.84) in dogs, and stertor (kappa, 0.65) and open-mouth breathing (kappa, 0.75) in cats, were considered sufficiently reliable among clinicians. Agreement on respiratory rate estimation was good in both species (intraclass correlation coefficient, 0.75). The greater the difference in clinical experience between two clinicians, the lower the odds of agreement between the two clinicians’ respiratory physical examination findings. Interclinician agreement was demonstrated to be poor for recognition of most respiratory clinical signs in dogs and cats. Teaching and clinical experience acquisition should be encouraged to improve respiratory clinical sign recognition.

Statistics From A (Agreement) to Z (z Score): A Guide to Interpreting Common Measures of Association, Agreement, Diagnostic Accuracy, Effect Size, Heterogeneity, and Reliability in Medical Research.

Researchers reporting results of statistical analyses, as well as readers of manuscripts reporting original research, often seek guidance on how numeric results can be practically and meaningfully interpreted. With this article, we aim to provide benchmarks for cutoff or cut-point values and to suggest plain-language interpretations for a number of commonly used statistical measures of association, agreement, diagnostic accuracy, effect size, heterogeneity, and reliability in medical research. Specifically, we discuss correlation coefficients, Cronbach's alpha, I2, intraclass correlation (ICC), Cohen's and Fleiss' kappa statistics, the area under the receiver operating characteristic curve (AUROC, concordance statistic), standardized mean differences (Cohen's d, Hedge's g, Glass' delta), and z scores. We base these cutoff values on what has been previously proposed by experts in the field in peer-reviewed literature and textbooks, as well as online statistical resources. We integrate, adapt, and/or expand previous suggestions in attempts to (a) achieve a compromise between divergent recommendations, and (b) propose cutoffs that we perceive sensible for the field of anesthesia and related specialties. While our suggestions provide guidance on how the results of statistical tests are typically interpreted, this does not mean that the results can universally be interpreted as suggested here. We discuss the well-known inherent limitations of using cutoff values to categorize continuous measures. We further emphasize that cutoff values may depend on the specific clinical or scientific context. Rule-of-the thumb approaches to the interpretation of statistical measures should therefore be used judiciously.

Le Fort I Osteotomy in Cleft Patients: Maxillary Advancement and Articulation.

Maxillary advancement may affect speech in cleft patients. To examine whether the amount of maxillary advancement and preoperativecephalometric skeletal and dentoalveolar relationships are associated with articulation errors of the Finnish alveolar consonants /s/, /l/, and /r/ in cleft patients. Fifty-seven nonsyndromic cleft patients who underwent Le Fort I or bimaxillary osteotomies were evaluated retrospectively. Pre- and post-operative lateral cephalometric radiographs and standardized speech video recordings were analyzed. The Aspin-Welch unequal variance t test, Student t test sign test, intraclass correlation and Kappa statistics were used in the statistical analyses. The mean advancement of the maxilla (point A) was 4.65 mm horizontally (range -2.80 to 11.30) and -3.82 mm vertically (range -14.20 to 3.90). The overall articulation (especially the sounds /s/ and /l/) improved significantly postoperatively, but the amount of maxillary advancement did not affect the articulation. The preoperative mean percentages of /s/, /l/, and /r/ errors were 32%, 33%, 46% and the postoperative percentages 23%, 19%, 40%, respectively. Preoperative articulation errors of /s/ were related to palatal inclination of the upper incisors. Orthognathic surgery may improve articulation errors. The amount of maxillary advancement is not related to the improvement.

How many trials are needed in kinematic analysis of reach-to-grasp?\u2014A study of the drinking task in persons with stroke and non-disabled controls

BackgroundKinematic analysis of the 3D reach-to-grasp drinking task is recommended in stroke rehabilitation research. The number of trials required to reach performance stability, as an important aspect of reliability, has not been investigated for this task. Thus, the aims of this study were to determine the number of trials needed for the drinking task to reach within-session performance stability and to investigate trends in performance over a set of trials in non-disabled people and in a sample of individuals with chronic stroke. In addition, the between-sessions test–retest reliability in persons with stroke was established.MethodsThe drinking task was performed at least 10 times, following a standardized protocol, in 44 non-disabled and 8 post-stroke individuals. A marker-based motion capture system registered arm and trunk movements during 5 pre-defined phases of the drinking task. Intra class correlation statistics were used to determine the number of trials needed to reach performance stability as well as to establish test–retest reliability. Systematic within-session trends over multiple trials were analyzed with a paired t-test.ResultsFor most of the kinematic variables 2 to 3 trials were needed to reach good performance stability in both investigated groups. More trials were needed for movement times in reaching and returning phase, movement smoothness, time to peak velocity and inter-joint-coordination. A small but significant trend of improvement in movement time over multiple trials was demonstrated in the non-disabled group, but not in the stroke group. A mean of 3 trials was sufficient to reach good to excellent test–retest reliability for most of the kinematic variables in the stroke sample.ConclusionsThis is the first study that determines the number of trials needed for good performance stability (non-disabled and stroke) and test–retest reliability (stroke) for temporal, endpoint and angular metrics of the drinking task. For most kinematic variables, 3–5 trials are sufficient to reach good reliability. This knowledge can be used to guide future kinematic studies.

Use of the Single-Item Self-Esteem Scale in a Juvenile Delinquent Population: A Convergent Validity Study

The validity of the Single-Item Self-Esteem Scale (SISE) in adolescents was not well established, and how self-esteem manifested in newly incarcerated juvenile delinquents was poorly described. Using a retrospective study with archival records, the SISE was compared to other measures of self-esteem, grit, mental health, and academic self-concept in a small juvenile detention center in the Midwest of the United States. Demographic data were analyzed, and a correlation, intraclass correlation, and kappa statistic were run to test relationships and reliability. The SISE was found to be a reliable and efficient tool to use with adolescents and juvenile delinquents. Policy recommendations applicable to juvenile delinquents and schools in general give direction using the findings.

Scale validation for the identification of falsified hand sanitizer: public and regulatory authorities perspectives from United Arab Emirates

BackgroundSince the time of declaration of global pandemic of COVID-19 by World Health Organization (WHO), falsified hand sanitizers surfaced regularly in markets, posing possible harm to public due to unlisted inclusion of methanol. The current research is an attempt to develop and validate a tool to document falsified hand sanitizer in the UAE community.MethodA descriptive cross-sectional community-based study was conducted among 1280 randomly selected participants. Respondents were sent a web-based electronic link to the survey via email. Content validity, factor analyses and known group validity were used to develop and validate a new scale to identify falsified hand sanitizer. Test-retest reliability, internal consistency, item internal consistency (IIC), and intraclass correlation coefficients (ICCs) were used to assess the reliability of the scale. SPSS version 24 was used to conduct data analysis.ResultsA total of 1280 participants were enrolled in the study. The content validity index (CVI) was 0.83 with the final scale of 12 items. The Kaiser-Meyer-Olkin (KMO) value was 0.788, with the Bartlett test of sphericity achieving statistical significance (p < 0.001). Our factor analysis revealed a 3-component model. The 3-factor solution was confirmed by PCFA analysis and had associations with good fit values. The PCFA for NFI was 0.970, CFI 0.978, and TLI 0.967. All values were in excess of 0.95, with RMSEA values below 0.06 at 0.03; all of these values indicated a good model fit. The Cronbach’s alpha was good overall (0.867). All factors had a Cronbach’s alpha value in excess of 0.70. The instrument demonstrated that every item met the IIC correlation standard ≥0.40. The scale displayed good overall ICC statistics of 0.867 (95% CI 0.856–0.877) with statistical significance (p < 0.001). The scale’s test-retest reliability was assessed through correlation of the falsified hand sanitizer identification score of respondents at the two time points. The test-retest correlation coefficient was 0.770 (p value < 0.01). Participants with post-graduate education were more likely to identify the falsified hand sanitizer compared to those with high school education. (p < 0.001).ConclusionsThis study developed and validated a new scale for the measurement of falsified hand sanitizer. This is expected to improve and promote collaboration between the health regulators and the public and hereby encourage customer satisfaction and participation.