Intraclass Correlation Coefficient Estimates Research Articles

Test-Retest Reliability of the STRAQ-1: A Registered Report

This Registered Report provides the first test of measurement invariance across time points and estimates of test-retest reliability for the Social Thermoregulation, Risk Avoidance Questionnaire (STRAQ-1, Vergara et al., 2019). The scale was developed and validated to understand the physiological drives underlying interpersonal bonding, measured by four constructs: the desire to socially regulate one’s temperature, the desire to solitary regulate one’s temperature, the sensitivity to higher temperatures, and the desire to avoid risk. Previous studies with large samples across 12 countries showed that the STRAQ-1 has a stable factorial structure, satisfying internal consistencies for the temperature subscales, and expected correlations in its nomological network. However, to date, this instrument has no estimates of test-retest reliability. Throughout four academic years (from 2018 to 2022), N = 183 French student participants took the STRAQ-1 at least two times. Out of the four STRAQ-1 subscales, only two were longitudinally invariant across two-time points. Thus, half of the STRAQ-1 constructs and latent scores were dissimilar and incomparable across time. We then conducted test-retest reliability using Intra Class Correlation coefficient (ICC) for the Social Thermoregulation, Solitary Thermoregulation, High-Temperature Sensitivity, and Risk Avoidance subscales. ICCs estimates were respectively for agreement and consistency: .70, .70 overall moderate to good, .62, .62 overall moderate, .67, .67 overall moderate, and .53, .53 overall poor to moderate, respectively. Our study suggests that test-retest reliability was insufficient for psychological diagnosis, and that future studies should address the problem of low generalizability of the constructs.

Development and validation of a virtual learning management system usability questionnaire: A case study.

The post-pandemic era has seen a surge in the popularity of Virtual Learning Management Systems (VLMS). However, there is a noticeable lack of tools to measure the usability of these systems. As technology evolves, user needs change, necessitating updated tools for system evaluation. This study aims to develop and validate a VLMS usability questionnaire, specifically designed to assess the usability of a university learning management system. The VLMS usability tool was systematically developed based on relevant domains identified in existing literature and expert opinions. It was then tested for face validity, content validity, and reliability. In a case study, the tool was distributed among 200 students from a Medical Sciences university who had used the Navid VLMS system. Semi-structured interviews with experts were analyzed using directed content analysis, resulting in 21 items categorized into four domains: effectiveness, reliability, learnability, and security. The content validity index and ratio were 0.939 and 0.976, respectively. The Intra Class Correlation (ICC) estimates for each section of the questionnaire ranged from 0.8-0.9, indicating high reliability. Cronbach's alpha was 0.97, suggesting excellent internal consistency. The case study results showed that the Navid platform achieved an average usability score of 70.36, with a standard deviation of 10.6, indicating moderate to high usability. The VLMS usability tool is a valid and reliable instrument for assessing the usability of the Navid learning management system. It can be used to improve the usability of the Navid system and serve as a benchmark for assessing the usability of other similar VLMSs.

Diagnostic agreement between visual examination and an automated scanner system with fluorescence for detecting and classifying occlusal carious lesions in primary teeth

ObjectivesTo investigate the in vivo diagnostic agreement between visual examination (VE) using the International Caries Detection and Assessment System (ICDAS) and an automated scanner system for detecting and classifying carious lesions in primary teeth. Methods5-year-old children (n = 216) underwent VE and intraoral scanning (TRIOS 4, 3Shape TRIOS A/S, Copenhagen, Denmark). Dental caries experience was recorded for each tooth surface using ICDAS. An automated, fluorescence-based caries scoring system was applied to eligible primary teeth occlusal surfaces on the 3D models using commercially available software. The automated system classified surfaces as sound, initial caries (ICDAS 01/02), or moderate-extensive caries (ICDAS ≥03). The diagnostic agreement was investigated using multi-level modelling and intraclass correlation coefficients. Analyses were repeated at both the initial threshold (ICDAS ≥01) and the moderate-extensive threshold (ICDAS ≥03). Results213 participants were included in the study, and 1525 primary molar occlusal surfaces were included in the analysis. The odds of detecting caries using the automated system were 46 % lower at the initial disease threshold (OR 0.54, 95 % CI 0.39–0.74) and 70 % lower at the moderate-extensive disease threshold (OR 0.30, 95 % CI 0.16–0.58) compared to VE. The intraclass correlation estimates at the initial and moderate-extensive thresholds were 0.90 (95 % CI 0.70–0.96) and 0.76 (95 % CI 0.22–0.94) respectively. ConclusionThe automated system is less likely to detect initial lesions and is more likely to underestimate lesion severity relative to visual examination using ICDAS. Clinical SignificanceClinically, using the automated tool to replace thorough visual inspection in primary teeth could result in missed opportunities to provide professional or self-care to arrest or reverse early disease. Additionally, it could misclassify moderate lesions as initial caries, potentially leading to complications associated with the delayed management of dental caries.

Advancing Consistency in Education: A Reliability Analysis of the Clinical Reasoning Assessment Tool.

Clinical reasoning (CR) is a fundamental component of physical therapist practice in health care. The Clinical Reasoning Assessment Tool (CRAT) was developed to assess CR in Doctor of Physical Therapy (DPT) and residency education. However, the reliability of the CRAT has not been investigated. Our purpose was to investigate the interrater and intrarater reliability of the CRAT. A range of assessment methods are available to assess CR, including didactic-based, simulation learning, and clinic-based assessments. The CRAT is designed to assess CR across learning environments. The CRAT assesses CR across 3 domains: (1) content knowledge, (2) procedural knowledge/psychomotor skills, and (3) conceptual reasoning. Twenty-two faculty participated. Participants completed an electronic survey at 2 timepoints (T1, T2) that included training on use of the CRAT and 2 case videos of DPT students at different performance levels managing a standardized patient. Participants rated student performance with the CRAT for both case videos (T1) and repeated the sequence at least 30 days later (T2). Intraclass correlation coefficient (ICC) estimates and 95% confidence intervals were used to determine the interrater and intrarater reliability for each domain of the CRAT. Video 1 was consistently rated higher for each CRAT domain (T1: 7.32-8.05; T2: 8.05-8.15) compared to video 2 (T1: 2.91-3.68; T2: 3.00-3.68). There were inverse relationships between CRAT ratings and number of years in DPT education, at an institution, and as a licensed physical therapist. At both timepoints, the interrater reliability for each domain was good to excellent (ICCs 0.60-0.79). The intrarater reliability was fair to excellent for all domains for each video (ICCs 0.45-0.75). The CRAT is a reliable instrument for assessing DPT student CR during a simulated patient encounter. Implementation of the CRAT in DPT education may facilitate improved consistency of CR assessment across the learner continuum.

Approaches for Quantifying the ICC in Multilevel Logistic Models: A Didactic Demonstration

Multilevel modeling techniques have gained traction among experimental psychologists for their ability to account for dependencies in nested data structures. Increasingly, these techniques are extended to the analysis of binary data (e.g., correct or incorrect responses). Despite their popularity, the information in logistic multilevel models is often underutilized when researchers focus solely on fixed effects and ignore important heterogeneity that exists between participants. In this tutorial, we review four techniques for estimating and quantifying the relative degree of between-person variability in logistic multilevel models in an accessible manner using real data. First, we introduce logistic multilevel modeling, including the interpretation of fixed and random effects. Second, we review the challenges associated with the estimation and interpretation of within- and between-participant variation in logistic multilevel models, particularly computing the intraclass correlation coefficient (ICC), which is usually a first, simple step in a linear MLM. Third, we demonstrate four existing methods of quantifying the ICC in logistic multilevel models and discuss their relative advantages and disadvantages. Fourth, we present bootstrapping methods to make statistical inference about these ICC estimates. To facilitate reuse, we developed R code to implement the discussed techniques, which is provided throughout the text and as supplemental materials.

KovaaK's aim trainer as a reliable metrics platform for assessing shooting proficiency in esports players: a pilot study.

Esports research lacks game-based metrics platforms appropriate for adequately capturing esports performance. The aim of this pilot study was to assess the reliability of the KovaaK's first-person shooter (FPS) aim trainer as a metrics platform for assessing shooting proficiency in esports players. Ten FPS esports players completed two identical experimental trials (T) separated by three to five days. Each trial included four rounds (R) of testing, evaluating four shooting tasks: Micro Flicking, Macro Flicking, Strafe Tracking, and Wall Peeking. Reliability of performance outcomes (e.g., accuracy, headshot accuracy, hits per second, and total shots hit) were assessed using the intraclass correlation coefficient (ICC) and their 95% confidence intervals (CI), and significant differences were identified using repeated-measures analysis of variance (RM-ANOVA). Results indicated excellent, or good to excellent reliability for all outcome variables with the ICC estimates ranging between 0.947-0.995, with lower and upper bound 95% CIs ranging between 0.876-0.988, and 0.984-0.999, respectively. Significant improvements were seen between experimental trials in the Macro Flicking task for accuracy (p = .005) and hits per second (p = .009) only. Significant interactions between trial and round were identified in the Micro Flicking task for accuracy (p = .006), with post hoc analysis showing accuracy was significantly higher in T1R1 compared to T2R1 (87.74 ± 3.13 vs. 85.99 ± 3.05, respectively, p = .02), and in T2R4 compared to T2R2 (87.99 ± 2.89 vs. 84.70 ± 4.25, respectively, p = .049). Significant interactions were also identified in the Strafe Tracking task for headshot accuracy (p = .002), with post hoc analysis showing headshot accuracy was significantly higher in T1R2 compared to T2R2 (78.48 ± 8.15 vs. 76.79 ± 12.16, respectively, p = .003), and in T1R2 compared to T1R1 (78.48 ± 8.15 vs. 73.68 ± 17.94, respectively, p = .023). In summary, this study demonstrates that KovaaK's provides a reliable metrics platform for assessing shooting proficiency in esports, however, some variability in performance was observed.

Concurrent validity and test-retest reliability of Squegg™ - smart dynamometer and handgrip trainer in healthy individuals.

Handgrip strength is crucial for overall health and is often evaluated using hand dynamometers. Two hand dynamometers were used in the study: the Jamar® Hydraulic Hand Dynamometer and Squegg™ Smart Dynamometer and Handgrip Trainer, which is a relatively new, Bluetooth-enabled device to measure handgrip strength. The purpose of the study was to compare Jamar and Squegg to determine concurrent validity and, separately, test-retest reliability of Squegg. The cross-sectional study measured bimanual handgrip strength using the Jamar and Squegg in two sessions, following standard protocols. Participants were 18+ years old, with no hand disease or dysfunction. Spearman's rank correlations were used to evaluate concurrent validity between Jamar and Squegg. Test-retest reliability was evaluated with intraclass correlation coefficient (ICC) estimates and 95% confidence intervals calculated using a mean-rating (k = 2) absolute agreement, 2-way mixed effects model. Data from 594 participants were used to evaluate concurrent validity. Moderately strong correlations between Jamar and Squegg values for the right hand (RH) (ρ = 0.670; p = .001) and Jamar and Squegg values for the left hand (LH) (ρ = 0.730; p = .001) were found, indicating 'good' concurrent validity. Test-retest reliability for Squegg ranged from good to excellent (RH, ICC = 0.911, 95% CI (0.895, 0.924), p = .001; LH, ICC = 0.928, 95% CI (0.915, 0.939), p = .001). The Squegg demonstrated good concurrent validity with Jamar and good-to-excellent test-retest reliability. It can be recommended as an alternative to the Jamar as a handgrip strength measurement tool. Squegg, which uses Bluetooth technology, is a reliable and cost-effective option for consistent handgrip strength testing.

Investigating Equivalence of In-Person and Telehealth-Based Neuropsychological Assessment Performance for Individuals Being Investigated for Younger Onset Dementia.

Given the rapid shift to in-home teleneuropsychology models, more research is needed to investigate the equivalence of non-facilitator models of teleneuropsychology delivery for people with younger onset dementia (YOD). This study aimed to determine whether equivalent performances were observed on neuropsychological measures administered in-person and via teleneuropsychology in a sample of people being investigated for YOD. Using a randomized counterbalanced cross-over design, 43 participants (Mage = 60.26, SDage = 7.19) with a possible or probable YOD diagnosis completed 14 neuropsychological tests in-person and via teleneuropsychology, with a 2-week interval. Repeated measures t-tests, intraclass correlation coefficients (ICC), and Bland Altman analyses were used to investigate equivalence across the administration conditions. No statistical differences were found between in-person and teleneuropsychology conditions, except for the Hospital Anxiety and Depression Scale Anxiety subtest. Small to negligible effect sizes were observed (ranging from .01 to .20). ICC estimates ranged from .71 to .97 across the neuropsychological measures. Bland Altman analyses revealed that the Wechsler Adult Intelligence Scale-Fourth Edition Block Design subtest had slightly better overall performance in the in-person condition and participants reported higher levels of anxiety symptoms during the teleneuropsychology condition; however, average anxiety symptoms remained within the clinically normal range. Participants reported a high level of acceptability for teleneuropsychology assessments. These results suggest that performances are comparable between in-person and teleneuropsychology assessment modalities. Our findings support teleneuropsychology as a feasible alternative to in-person neuropsychological services for people under investigation of YOD, who face significant barriers in accessing timely diagnoses and treatment options.

Within-subject variation of C-reactive protein and high-sensitivity C-reactive protein: A systematic review and meta-analysis.

C-reactive protein (CRP) and high-sensitivity C-reactive protein (hsCRP) are measures of inflammation used in diagnosis, to guide treatment decisions, and in disease prediction. Variability in measured CRP and hsCRP may affect their clinical utility but estimates of within-subject variability are based on limited data. A systematic review and meta-analysis was performed to estimate longitudinal within-subject variability of CRP and hsCRP over any time period. Follow-up studies of any design in adults or children, with repeated measures of CRP or hsCRP were sought. Multiple databases were searched from inception to November 2022. Titles and abstracts were screened in duplicate. Full text screening and data extraction were performed by one reviewer and verified by a second. Risk of bias was assessed with a modified Consensus-based Standards for the Selection of Health Measurement Instruments (COSMIN) tool. Intraclass correlation coefficient (ICC) results were pooled with a meta-analysis and coefficient of variation (CV) results were described by median and range. Of 2675 studies identified, 60 met the inclusion criteria: 34 reported CRP and 26 reported hsCRP. For CRP, median CV was 0.41 (range 0.11 to 0.89), and the pooled estimate of ICC was 0.55 (95% CI 0.35 to 0.74). For hsCRP, median CV was 0.44 (range 0.27 to 0.76) and the pooled estimate of ICC was 0.62 (95% CI 0.58 to 0.67). Assessment of variability was not the main aim of many of the included papers, and it is possible that some relevant papers have been missed. Many of the papers included had low numbers of participants and/or low numbers of repeated measurements. Estimated within-subject variability is high for both CRP and hsCRP, but estimates are based on small numbers of participants and measurements. There is a need for better estimates of within-subject variability from analysis of larger numbers of repeated measurements in larger numbers of subjects.

A Validation Study of the Mindfulness-Based Interventions Teaching Assessment Criteria for Assessing Mindfulness-Based Intervention Teacher Skill: Inter-Rater Reliability and Predictive Validity.

Prior data suggests the Mindfulness-Based Interventions: (MBI) Teaching Assessment Criteria (MBI:TAC) has good inter-rater reliability, but many raters knew teacher experience level. We sought to further evaluate the MBI-TAC's inter-rater reliability and obtain preliminary data on predictive validity. We videorecorded 21 MBSR teachers from academic and community settings. We trained 19 experienced MBI teachers in using the MBI:TAC. MBSR teachers were rated by three assessors; teachers and their assessors did not know one another. To assess predictive validity, MBSR students in courses taught by 18 of the MBSR teachers were invited to complete PROMIS-29 measures before the MBSR course, at the end of the course (month 2), and month 4. Intraclass correlation coefficients (ICCs) representing a single rater ranged from 0.33 to 0.56 on the 6 MBI:TAC domains. Using an average of two raters, ICC estimates ranged from 0.48 to 0.71 and ICCs generalizing to an average of three raters ranged from 0.6 to 0.8. Among n = 152 participating MBSR students, we found improvements from baseline to 2months and 4months in PROMIS measures of Anxiety, Depression, Fatigue, Sleep, and Social Role function (range in improvement 2.3 to 6.3, P < 0.0001 for all comparisons except Social Role at 2months, P = 0.007). Higher MBI:TAC ratings were associated with greater improvements in anxiety among MBSR students from baseline to 2months, with a -0.31 lower participant anxiety score per 1 unit increase in MBI:TAC composite teaching rating (95% CI -0.58, -0.05, P = 0.019), but we did not find statistically significant relationships with improvements in other PROMIS-29 domains. ICCs indicated good reliability using an average of three ratings, but inter-rater reliability was only fair using a single rater. We found initial validation that higher MBI:TAC ratings predicted greater improvements in anxiety symptoms in MBSR participants.

Estimating individual health-related quality of life changes in low back pain patients

BackgroundThere is a need to evaluate different options for estimating individual change in health-related quality of life for patients with low back pain.MethodsSecondary analysis of data collected at baseline and 6 weeks later in a randomized trial of 749 adults with low back pain receiving usual medical care (UMC) or UMC plus chiropractic care at a small hospital at a military training site or two large military medical centers. The mean age was 31; 76% were male and 67% were White. The study participants completed the Patient-Reported Outcomes Measurement Information System (PROMIS®)-29 v 1.0 physical function, pain interference, pain intensity, fatigue, sleep disturbance, depression, anxiety, satisfaction with participation in social roles, physical summary, and mental health summary scores (T-scored with mean = 50 and standard deviation (SD) = 10 in the U.S. general population).ResultsReliability estimates at the baseline ranged from 0.700 to 0.969. Six-week test–retest intraclass correlation estimates were substantially lower than these estimates: the median test–retest intraclass correlation for the two-way mixed-effects model was 0. 532. Restricting the test–retest reliability estimates to the subset who reported they were about the same as at baseline on a retrospective rating of change item increased the median test–retest reliability to 0.686. The amount of individual change that was statistically significant varied by how reliability was estimated, and which SD was used. The smallest change needed was found when internal consistency reliability and the SD at baseline were used. When these values were used, the amount of change needed to be statistically significant (p < .05) at the individual level ranged from 3.33 (mental health summary scale) to 12.30 (pain intensity item) T-score points.ConclusionsWe recommend that in research studies estimates of the magnitude of individual change needed for statistical significance be provided for multiple reliability and standard deviation estimates. Whenever possible, patients should be classified based on whether they 1) improved significantly and perceived they got better, 2) improved significantly but did not perceive they were better, 3) did not improve significantly but felt they got better, or 4) did not improve significantly or report getting better.

Evaluation of laser power stability of repeatedly used SubCyclo probe in micropulse transscleral cyclophotocoagulation for glaucoma: A step towards sustainable ophthalmic surgery

Purpose To evaluate the laser power stability of the SubCyclo probe for micropulse transscleral cyclophotocoagulation after repeated use. Materials and methods This experimental study involved 6 new probes. Each probe was connected to the SubCyclo mode (2,000 mW power, 31.3% duty cycle, and 100 seconds duration) of the Vitra 810 laser delivery system (Quantel Medical, France). Laser power measurements were taken using a calibrated laser power meter (Nova, Ophir Optronics Solutions, Israel) every 10 seconds from 10 to 90 seconds during each of the 40 cycles. Intra-rater reliability was assessed using intraclass correlation (ICC). A linear mixed model for repeated measures and pairwise comparisons with Bonferroni adjustment were used for the analysis. Results The mean (SD) power outputs of all probes for the first cycle and all cycles were 421.9 (19.7) mW and 436.7 (16.1) mW, respectively. During the first cycle, the mean (SD) laser power gradually decreased from 444.3 (13.4) mW at 10 seconds to 407.3 (17.0) mW at 90 seconds (Fig 3). For all cycles, the power was 446.0 (13.6) mW at 10 seconds and gradually declined to 426.8 (21.0) mW at 90 seconds. Pairwise comparisons revealed significant differences in mean laser power outputs after 16 cycles of repeated use compared to the first cycle. The ICC estimate (95% CI) for intra-rater reliability was 0.96 (0.89, 0.99). Conclusions The SubCyclo probe maintains stable laser power outputs throughout repeated use for up to 16 cycles, with a significant increase observed after 16 cycles.

The level of agreement between the numerical rating scale and visual analogue scale for assessing pain intensity in adults with chronic pain.

The numerical rating scale and visual analogue scale are used to quantify pain intensity. However, it has not yet been explored whether these scores are interchangeable in adults with chronic pain. Data from the prospective multicentre cross-sectional INTERVAL study were used to evaluate the one-dimensionality and agreement between numerical rating scale scores and visual analogue scale scores in adults with chronic pain. Pain intensity scores using the numerical rating scale and visual analogue scale were provided by 366 patients with chronic pain for current, average, minimal and maximal pain. To evaluate whether pain intensity scales are completed in accordance with each other, the proportion of patients who satisfied the following condition was calculated: minimal pain intensity ≤ maximal pain intensity. A factor analysis confirmed the one-dimensionality of the pain measures. A significant difference was found between numerical rating scale and visual analogue scale scores for average, current, minimum and maximum pain. Intra-class correlation coefficient estimates ranged from 0.739 to 0.858 and all measures failed to show sufficient and acceptable agreement at the 95% level. The strength of agreement between pain severity categories was classified as 'moderate' for average and minimal pain and 'substantial' for current and maximal pain. The proportion of patients who scored minimal pain ≤ maximal pain was 97.5% for the numerical rating scale and 89.5% for the visual analogue scale. This study failed to show an acceptable agreement between the numerical rating scale and visual analogue scale when pain intensity was rated by adults with chronic pain, despite showing both scales measure the same information.

Within-individual variation of measured depression symptoms: A systematic review and meta-analysis

BackgroundDepression screening instruments are commonly used to assess the presence and severity of depression symptoms. However, there is little information on variation in depression screening scores over time within the same individual. MethodsA systematic review and meta-analysis was performed of studies reporting the within-subject variability of the Beck's Depression Inventory (BDI), Hospital Anxiety and Depression Scale (HADS), Patient Health Questionnaire-9 (PHQ-9) and Patient Health Questionnaire-2 (PHQ-2). Multiple databases were searched from inception to 14th July 2023. Title and abstract screening was performed in duplicate, full text screening and data extraction by one reviewer and verified by a second. Risk of bias was assessed with a modified COSMIN tool. ResultsOf 2798 titles and abstracts and 157 full text articles screened, 41 met the inclusion criteria. No studies were on patients with depression, most had only two measurements, less than two weeks apart. The pooled estimates of ICCs (Intraclass Correlations) and 95 % confidence intervals for BDI, HADS, PHQ-9 and PHQ-2 were 0.89 (0.84–0.93), 0.89 (0.85–0.93), 0.84 (0.81–0.88) and 0.75 (0.56–0.93), respectively.Studies in healthy subgroups showed lower ICC than those with physical illness. LimitationsAssessment of variability is not the main aim of most papers assessing depression measurement instruments, so it is possible that some relevant papers have been missed. ConclusionsWithin-subject scores for BDI, HADS and PHQ-9, show generally excellent agreement over periods of up to two weeks. However published data on within-subject variability is lacking over longer time periods and for patients with depression.

Alternative Endoscopy Reading Paradigms Determine Score Reliability and Effect Size in Ulcerative Colitis.

Central reading of endoscopy is advocated by regulatory agencies for clinical trials in ulcerative colitis [UC]. It is uncertain whether the local/site reader should be included in the reading paradigm. We explore whether using locally- and centrally-determined endoscopic Mayo subscores [eMS] provide a reliable final assessment and whether the paradigm used has an impact on effect size. eMS data from the TURANDOT [NCT01620255] study were used to retrospectively examine seven different reading paradigms (using the scores of local readers [LR], first central readers [CR1], second central readers [CR2], and various consensus reads [ConCR]) by assessing inter-rater reliabilities and their impact on the key study endpoint, endoscopic improvement. More than 40% of eMS scores between two trained central readers were discordant. Central readers had wide variability in scorings at baseline (intraclass correlation coefficient [ICC] of 0.475 [0.339, 0.610] for CR1 vs CR2). Centrally-read scores had variable concordance with LR (LR vs CR1 ICC 0.682 [0.575, 0.788], and LR vs CR2 ICC 0.526 [0.399, 0.653]). Reading paradigms with LR and CR which included a consensus, enhanced ICC estimates to >0.8. At Week 12, without the consensus reads, the CR1 vs CR2 ICC estimates were 0.775 [0.710, 0.841], and with consensus reads the ICC estimates were >0.9. Consensus-based approaches were most favourable to detect a treatment difference. The ICC between the eMS of two trained and experienced central readers is unexpectedly low, which reinforces that currently used central reading processes are still associated with several weaknesses.

The role of sound touch elastography in assessment of liver fibrosis in chronic liver disease keeping APRI as the reference standard.

The study aims to determine the role of Sound Touch Elastography [STE] technique in staging liver fibrosis and predicting clinically significant gastro-esophageal varices among patients with chronic liver disease [CLD] keeping aspartate aminotransferase to platelet ratio index [APRI] as the reference standard. A prospective short-term study including 60 eligible patients with CLD were staged as non-significant fibrosis [NSF], significant fibrosis [SF] and cirrhosis [C] based on APRI values. STE was performed on each patient obtaining multiple readings as per pre-defined standards. The intra-observer reliability between each measurement and its association with APRI staging was evaluated using relevant statistical variables. Further, Youden's index was used to define the optimum cut-off values on STE in differentiating the stages of fibrosis and in predicting clinically significant gastro-esophageal varices. Based on APRI cut-off values, 41.7% [n=25] of the study population had cirrhosis, while 45% [n=27] had significant fibrosis and 13.3% [n=8] had NSF. The STE values in kPa showed a positive correlation with APRI values [(rs)=0.837, p<0.001]. The intra-class correlation estimates based on a mean rating [k=5] was found to be 0.97 [0.95-0.99], implying an excellent agreement between the measurements. Optimum cut-off values in staging SF and C were 7.26 kPa [J=0.73, sensitivity-85.19%, specificity-87.5%; 95% CI] and 13.79 kPa [J=0.84, sensitivity-96.0%, specificity-88.89%; 95% CI]. The AUROC for each of these stages were 0.926 [0.785-0.987] and 0.976 [0.890-0.999], respectively. 23.3% [n=14] of the study population had clinically significant gastro-esophageal varices with a value above 18.84 kPa [J=0.88] showing a sensitivity of 92.85% and a specificity of 95.65% in predicting the same. The novel STE technique shows good accuracy in staging liver fibrosis as determined by APRI values and in prediction of clinically significant gastro-esophageal varices with excellent reliability. It shows promising prospects and can be integrated widely in clinical practice for assessment and staging of fibrosis in CLD.

An Application of Theril Indexes for the Interrater Reliability: A Comparison with Intraclass Correlations

This study proposes to apply the Theil-index ratios for the interrater reliability. We discuss the theoretical foundations and examine its function empirically using real data. Our analyses show that Theil-index rations and intraclass correlation (ICC) estimates are highly correlated. However, ICC may underestimate the interrater reliability by some extreme disagreement among raters and be more likely to be influenced by the extreme disagreement. As Theil-index ratios overcome the limitations of ICC to some degree, it seems that Theil-index ratios provide an alternative to evaluating interrater reliability, at least under certain conditions, e.g., when outliers exist in the data, it is difficult to obtain the variance component estimates, or ICC underestimates interrater reliability.

Exploring the usefulness of the Brief COPE in clinical and positive psychology: A discriminant content validity study

The Brief Coping Orientation to Problem Experienced (COPE) scale is widely used for measuring coping strategies. However, concerns persist regarding the dimensions captured by the scale and the accuracy of item representation. This study examined the relevance of adapted Brief COPE items using discriminant content validity (DCV). A panel of experts (n = 15) assessed the extent to which the items corresponded to their intended dimensions. Intraclass correlation (ICC) estimates ranged from 0.640 to 0.828, indicating agreement among experts. A one-sample t-test evaluated DCV, revealing that 21 out of 28 items distinctly and exclusively measured intended dimensions, confirming their discriminant content validity. Seven items were excluded: three did not measure coping strategies (non-dimension), and four measured them in different dimensions (wrong-dimension). The discriminant content-validated Brief COPE scale improves coping assessment, benefiting psychological therapies and providing researchers with refined measures for each coping strategy dimension, addressing dimensional concerns.

Reliability of the Modified Method for Sphericity Deviation Score Using Only the Involved Hip Radiographs in the Patients With Legg-Calve-Perthes Disease.

The sphericity deviation score (SDS) is a validated radiographic outcome measure that quantifies the severity of the femoral head deformity in the healed stage of Legg-Calve-Perthes disease (LCPD). The current method requires radiographs of both hips regardless of unilateral involvement to standardize radiographic magnification. As LCPD affects the unilateral hip in 85% to 90% of cases, the current method imposes unnecessary radiation exposure to most of the patients and having to exclude patients from research studies who only had unilateral hip radiographs. We thus modified the SDS method to use unilateral hip radiographs. The purpose of this study was to investigate the reliability of the modified SDS method using the radiographs containing only one hip. This retrospective study involved 40 patients with LCPD with unilateral involvement in the healed stage of LCPD. We modified the SDS measurement method by using the distance between the teardrop and the lateral acetabulum for magnification correction and providing a clear anatomic description of reference points on the femoral head. Three independent observers performed measurements using radiographs containing the affected hip only (modified method) and both hips (conventional method). The intraclass correlation (ICC) estimates were calculated. To verify clinical relevance, the correlation of the SDS with the Stulberg classification and hip range of motion (ROM) were also investigated. Measurements using the modified SDS showed excellent inter-/intra-observer ICCs (0.903 to 0.978). The ICCs between the modified and conventional methods were also excellent: 0.940 to 0.966 within the same observer and 0.897 to 0.919 between different observers. The modified SDS showed moderate-to-strong correlations with the Stulberg classification (Spearman ρ=0.650) and hip ROM (Pearson r =-0.661). The modified SDS measurement method showed excellent inter-/intra-observer reliability and moderate-to-strong correlations with the Stulberg classification and hip ROM. This method will help reduce unnecessary radiation exposure in patients with unilateral LCPD and prevent the exclusion of patients with unilateral radiographs from future research studies. Level III-diagnostic study.

A systematic evaluation of normalization methods and probe replicability using infinium EPIC methylation data

BackgroundThe Infinium EPIC array measures the methylation status of > 850,000 CpG sites. The EPIC BeadChip uses a two-array design: Infinium Type I and Type II probes. These probe types exhibit different technical characteristics which may confound analyses. Numerous normalization and pre-processing methods have been developed to reduce probe type bias as well as other issues such as background and dye bias.MethodsThis study evaluates the performance of various normalization methods using 16 replicated samples and three metrics: absolute beta-value difference, overlap of non-replicated CpGs between replicate pairs, and effect on beta-value distributions. Additionally, we carried out Pearson’s correlation and intraclass correlation coefficient (ICC) analyses using both raw and SeSAMe 2 normalized data.ResultsThe method we define as SeSAMe 2, which consists of the application of the regular SeSAMe pipeline with an additional round of QC, pOOBAH masking, was found to be the best performing normalization method, while quantile-based methods were found to be the worst performing methods. Whole-array Pearson’s correlations were found to be high. However, in agreement with previous studies, a substantial proportion of the probes on the EPIC array showed poor reproducibility (ICC < 0.50). The majority of poor performing probes have beta values close to either 0 or 1, and relatively low standard deviations. These results suggest that probe reliability is largely the result of limited biological variation rather than technical measurement variation. Importantly, normalizing the data with SeSAMe 2 dramatically improved ICC estimates, with the proportion of probes with ICC values > 0.50 increasing from 45.18% (raw data) to 61.35% (SeSAMe 2).