Intraclass Correlation Coefficient Estimates Research Articles

INTERRATER RELIABILITY OF AN ADVANCED NONCONTACT WOUND MEASUREMENT SYSTEM USING ARTIFICIAL INTELLIGENCE

Abstract Venous leg ulcers (VLU) are recurring, disabling wounds that develop primarily in older adults with comorbidities. In an ongoing clinical trial testing a nutritional intervention researchers use advanced technology (Swift Skin and Wound) to measure VLU healing over time. This noncontact wound measurement system uses artificial intelligence to automatically calculate wound measurements and track healing progress. However, a subjective component of the Swift system involves delineating wound perimeters on photographs captured with a tablet or smartphone. To evaluate interrater reliability of the system in the current study, measurements of 11 wounds by two independent raters were assessed using an intraclass correlation coefficient (ICC). ICC estimates and their 95% confidence intervals were calculated using SPSS statistical package version 27 based on a mean-rating (k = 2), consistency, 2-way mixed-effects model. For area measures, ICC = 0.99 with 95% confidence interval = 0.998-1.0, indicating the Swift measurement system has excellent interrater reliability.

Reliability of the ISAKOS Modification to Subclassify Rockwood Type III Acromioclavicular Joint Injuries.

To improve clinical management, the International Society of Arthroscopy, Knee Surgery and Orthopaedic Sports Medicine (ISAKOS) developed a specific subclassification of Rockwood type III acromioclavicular (AC) injuries: IIIA (no overriding distal clavicle) and IIIB (overriding distal clavicle). The study aimed to determine the inter- and intraobserver reliability of the radiographic classification proposed by ISAKOS for AC injuries. It was hypothesized that the strength of agreement for the ISAKOS modification will be substantial to almost perfect. Cohort study (diagnosis); Level of evidence, 3. We evaluated 40 radiographs of all types of AC joint injuries from patients at a single institution. The images were distributed to 6 shoulder and elbow fellowship-trained orthopaedic surgeons, along with standardized assessment questionnaires. The evaluators measured the bilateral coracoclavicular distance and the coracoclavicular ratio and classified the severity of the injuries according to the modified ISAKOS Rockwood classification. Four of the surgeons repeated the evaluation 6 weeks later to calculate intraobserver agreement. The kappa (κ) statistic was calculated for categorical inter- and intraobserver reliability. Intraclass correlation coefficient (ICC) estimates were calculated for the reliability of the coracoclavicular distance measurement. A 2-tailed test was performed to assess statistical significance. Overall interobserver agreement was substantial (κ = 0.637; 95% CI, 0.595-0.680) among the 6 evaluators, and intraobserver agreement was also substantial among the 4 evaluators (κ = 0.616; 95% CI, 0.549-0.638). The interobserver agreement for evaluating types IIIA and IIIB was fair (κ = 0.215; 95% CI, 0.135-0.295) and moderate (κ = 0.473; 95% CI, 0.393-0.553), respectively. The agreement on the coracoclavicular distance measurements was excellent among the 6 evaluators on both the affected side (ICC, 0.982; 95% CI, 0.970-0.990) and the unaffected side (ICC, 0.930; 95% CI, 0.894-0.958). Substantial agreement was found when categorizing AC joint injuries using the ISAKOS modification of the Rockwood classification, with excellent reliability demonstrated for the quantitative assessment of vertical displacement of the clavicle. Visual examination was unreliable in differentiating type IIIA injuries from type IIIB injuries.

Sample size recommendations for studies on reliability and measurement error: an online application based on simulation studies

Simulation studies were performed to investigate for which conditions of sample size of patients (n) and number of repeated measurements (k) (e.g., raters) the optimal (i.e., balance between precise and efficient) estimations of intraclass correlation coefficients (ICCs) and standard error of measurements (SEMs) can be achieved. Subsequently, we developed an online application that shows the implications for decisions about sample sizes in reliability studies. We simulated scores for repeated measurements of patients, based on different conditions of n, k, the correlation between scores on repeated measurements (r), the variance between patients’ test scores (v), and the presence of systematic differences within k. The performance of the reliability parameters (based on one-way and two-way effects models) was determined by the calculation of bias, mean squared error (MSE), and coverage and width of the confidence intervals (CI). We showed that the gain in precision (i.e., largest change in MSE) of the ICC and SEM parameters diminishes at larger values of n or k. Next, we showed that the correlation and the presence of systematic differences have most influence on the MSE values, the coverage and the CI width. This influence differed between the models. As measurements can be expensive and burdensome for patients and professionals, we recommend to use an efficient design, in terms of the sample size and number of repeated measurements to come to precise ICC and SEM estimates. Utilizing the results, a user-friendly online application is developed to decide upon the optimal design, as ‘one size fits all’ doesn’t hold.

Use and repeatability of 3D light scanning to measure transverse dorsal profile size and symmetry in the thoracic region in horses

Equine epaxial muscle size, thoracolumbar profile and symmetry in horses is of clinical interest due to relationships with pain and pathology. Flexible-curve rulers have previously been used to gather reliable, objective measures of thoracic profile, however, 3D light-scanning offers a potential non-contact alternative method to estimate cross sectional area (CSA) of the region. 3D light-scans of the thoracic epaxial region were taken from ten endurance horses (7 geldings, 3 mares; 8±2 years). Total CSA of the combined epaxial musculature, using computer software, was calculated at scapula and T18 levels (depth: 15 cm). Intra and inter-rater (n=3) reliability of CSA measurements was assessed using Friedman’s analyses and post-hoc Wilcoxon rank tests (three repeated measures). Intraclass correlation estimates (ICC ± 95% confidence intervals (CI)) were calculated (mean-rating, absolute-agreement, 2-way mixedeffects model). Paired t-tests assessed differences between right and left areas. No significant differences existed for transverse plane-cuts (scapula, T18 P>0.05) between light-scans. Right and left areas were significantly different at the withers (P=0.012) with the left side larger in 70% of scans, but no significant differences were found between sides at T18. No differences existed for different plane-cuts of the same horse (P=0.53; ICC: 0.76; CIs: 0.43-0.92). While reliability was reduced between all raters (P=0.02; ICC: 0.70; CIs: 0.56-0.82), no significant differences occurred between two different assessors experienced in using the software (P=0.88; ICC: 0.90; CIs: 0.82-0.95). Intra-rater reliability for assessing thoracic profile and inter-rater reliability ICC values with experienced analysts was interpreted as good/excellent. The results suggest 3D light-scanning is an objective, non-invasive method to record size and symmetry of the epaxial region in horses and warrants validity testing against current measurement methods such as the flexible-curve ruler.

Non-Expert Markings of Active Chronic Graft-Versus-Host Disease Photographs: Optimal Metrics of Training Effects.

Lack of reliable measures of cutaneous chronic graft-versus-host disease (cGVHD) remains a significant challenge. Non-expert assistance in marking photographs of active disease could aid the development of automated segmentation algorithms, but validated metrics to evaluate training effects are lacking. We studied absolute and relative error of marked body surface area (BSA), redness, and the Dice index as potential metrics of non-expert improvement. Three non-experts underwent an extensive training program led by a board-certified dermatologist to mark cGVHD in photographs. At the end of the 4-month training, the dermatologist confirmed that each trainee had learned to accurately mark cGVHD. The trainees' inter- and intra-rater intraclass correlation coefficient estimates were "substantial" to "almost perfect" for both BSA and total redness. For fifteen 3D photos of patients with cGVHD, the trainees' median absolute (relative) BSA error compared to expert marking dropped from 20 cm2 (29%) pre-training to 14 cm2 (24%) post-training. Total redness error decreased from 122 a*·cm2 (26%) to 95 a*·cm2 (21%). By contrast, median Dice index did not reflect improvement (0.76 to 0.75). Both absolute and relative BSA and redness errors similarly and stably reflected improvements from this training program, which the Dice index failed to capture.

Intraclass Correlation and Variance in the Characteristics of Primary Care Patients Managing Chronic Medical and Behavioral Conditions.

Background To avoid statistical errors, researchers who recruit patients from selected medical practices and analyze them at the individual level need to account for the clustered nature of their sample. This is most often done using the intraclass correlation coefficients (ICCs), a measure of how strongly subjects recruited from the same cluster (in this case patients from a clinic) resemble each other. Aims The aim is to support the design of cluster-randomized studies by supplying estimates of variance and ICC of various measures using a population of patients from multiple primary care clinics. Materials and methods ICCs were extracted from a large cluster-randomized pragmatic clinical trial of adult primary care patients managing multiple chronic conditions, the Integrating Behavioral Health and Primary Care study (IBH-PC). IBH-PC collected demographics and patient-reported health outcomes on over 3,000 adults from 44 primary care practices in 13 states across the US. We present estimates of the standard deviation and ICC for gender, race, ethnicity, marital status, employment, income, education, social determinants of health, PROMIS-29 functional status, Duke Activity Status Index (DASI), nine-item Patient Health Questionnaire (PHQ-9) depression score, Generalized Anxiety Disorder (GAD-7) anxiety score, Asthma Symptom Utility Index, restricted activity days, medication adherence, health care visits in the past month, emergency room visits in the past year, hospital days in the past year, perception of quality and patient-centeredness of care, alcoholic drinks per month, and the GAIN substance use disorder screener. Results ICCs varied broadly with the highest values found for race and income and the lowest for short-term estimates of the GAIN. Conclusions These values can be used to inform the design, especially power estimates and sample size requirements, of future studies.

IccCounts: An R Package to Estimate the Intraclass Correlation Coefficient for Assessing Agreement with Count Data

The intraclass correlation coefficient (ICC) is a widely used index to assess agreement with continuous data. The common approach for estimating the ICC involves estimating the variance components of a linear mixed model under assumptions such as linearity and normality of effects. However, if the outcomes are counts these assumptions are not met and the ICC estimates are biased and inefficient. In this situation, it is necessary to use alternative approaches that are better suited for count data. Here, the iccCounts R package is introduced for estimating the ICC under the Poisson, Negative Binomial, Zero-Inflated Poisson and Zero-Inflated Negative Binomial distributions. The utility of the iccCounts package is illustrated by three examples that involve the assessment of repeatability and concordance with count data.

T2 Turbo Spin Echo With Compressed Sensing and Propeller Acquisition (Sampling k-Space by Utilizing Rotating Blades) for Fast and Motion Robust Prostate MRI: Comparison With Conventional Acquisition.

The aim of this study was to compare a new compressed sensing (CS) method for T2-weighted propeller acquisitions (T2 CS ) with conventional T2-weighted propeller sequences (T2 conv ) in terms of achieving a higher image quality, while reducing the acquisition time. Male participants with a clinical suspicion of prostate cancer were prospectively enrolled and underwent prostate magnetic resonance imaging at 3 T. Axial and sagittal images of the T2 conv sequence and the T2 CS sequence were acquired. Sequences were qualitatively assessed by 2 blinded radiologists concerning artifacts, image-sharpness, lesion conspicuity, capsule delineation, and overall image quality using 5-point Likert items ranging from 1 (nondiagnostic) to 5 (excellent). The apparent signal-to-noise ratio and apparent contrast-to-noise ratio were evaluated. PI-RADS scores were assessed for both sequences. Statistical analysis was performed by using Wilcoxon signed rank test and paired samples t test. Intrarater and interrater reliability of qualitative image evaluation was assessed using intraclass correlation coefficient (ICC) estimates. A total of 29 male participants were included (mean age, 66 ± 8 years). The acquisition time of the T2 CS sequence was respectively 26% (axial plane) and 24% (sagittal plane) shorter compared with the T2 conv sequence (eg, axial: 171 vs 232 seconds; P < 0.001). In the axial plane, the T2 CS sequence had fewer artifacts (4 [4-4.5] vs 4 [3-4]; P < 0.001), better image-sharpness (4 [4-4.5] vs 3 [3-3.5]; P < 0.001), better capsule delineation (4 [3-4] vs 3 [3-3.5]; P < 0.001), and better overall image quality (4 [4-4] vs 4 [3-4]; P < 0.001) compared with the T2 conv sequence. The ratings of lesion conspicuity were similar (4 [4-4] vs 4 [3-4]; P = 0.166). In the sagittal plane, the T2 CS sequence outperformed the T2 conv sequence in the categories artifacts (4 [4-4] vs 3 [3-4]; P < 0.001), image sharpness (4 [4-5] vs 4 [3-4]; P < 0.001), lesion conspicuity (4 [4-4] vs 4 [3-4]; P = 0.002), and overall image quality (4 [4-4] vs 4 [3-4]; P = 0.002). Capsule delineation was similar between both sequences (3 [3-4] vs 3 [3-3]; P = 0.07). Intraobserver and interobserver reliability for qualitative scoring were good (ICC intra: 0.92; ICC inter: 0.86). Quantitative analysis revealed a higher apparent signal-to-noise ratio (eg, axial: 52.2 ± 9.7 vs 22.8 ± 3.6; P < 0.001) and a higher apparent contrast-to-noise ratio (eg, axial: 44.0 ± 9.6 vs 18.6 ± 3.7; P ≤ 0.001) of the T2 CS sequence. PI-RADS scores were the same for both sequences in all participants. CS-accelerated T2-weighted propeller acquisition had a superior image quality compared with conventional T2-weighted propeller sequences while significantly reducing the acquisition time.

Reproducibility of the Blood and Urine Exposome: A Systematic Literature Review and Meta-Analysis.

Endogenous and exogenous metabolite concentrations may be susceptible to variation over time. This variability can lead to misclassification of exposure levels and in turn to biased results. To assess the reproducibility of metabolites, the intraclass correlation coefficient (ICC) is computed. A literature search in three databases from 2000 to May 2021 was conducted to identify studies reporting ICCs for blood and urine metabolites. This review includes 192 studies, of which 31 studies are included in the meta-analyses. The ICCs of 359 single metabolites are reported, and the ICCs of 10 metabolites were meta-analyzed. The reproducibility of the single metabolites ranges from poor to excellent and is highly compound-dependent. The reproducibility of bisphenol A (BPA), mono-ethyl phthalate (MEP), mono-n-butyl phthalate (MnBP), mono-2-ethylhexyl phthalate (MEHP), mono(2-ethyl-5-hydroxyhexyl) phthalate (MEHHP), mono-benzyl phthalate (MBzP), mono-(2-ethyl-5-oxohexyl) phthalate (MEOHP), methylparaben, and propylparaben is poor to moderate (ICC median: 0.32; range: 0.15-0.49), and for 25-hydroxyvitamin D [25(OH)D], it is excellent (ICC: 0.95; 95% CI, 0.90-0.99). Pharmacokinetics, mainly the half-life of elimination and exposure patterns, can explain reproducibility. This review describes the reproducibility of the blood and urine exposome, provides a vast dataset of ICC estimates, and hence constitutes a valuable resource for future reproducibility and clinical epidemiologic studies.

Defining an Optimal Sample Size for Corneal Epithelial Immune Cell Analysis Using in vivo Confocal Microscopy Images.

PurposeIn vivo confocal microscopy (IVCM) images are frequently used to quantify corneal epithelial immune cell (IC) density in clinical studies. There is currently limited evidence to inform the selection of a representative image sample size to yield a reliable IC density estimate, and arbitrary numbers of images are often used. The primary aim of this study was to determine the number of randomly selected, unique IVCM images required to achieve an acceptable level of accuracy when quantifying epithelial IC density, in both the central and peripheral cornea. The secondary aim was to evaluate the consistency and precision of an image selection approach where corneal epithelial IC density was quantified from “three representative images” selected independently by three experienced observers.MethodsAll combinations of two to 15 non-overlapping IVCM images were used for deriving IC density estimates, for both the central and peripheral cornea, in 20 healthy participants; the density value from averaging quantifications in the 16 images was defined as the “true mean”. IC density estimates were compared with the true mean in each corneal region using a mean ratio. Intraclass correlation coefficients (ICCs) were used to evaluate the consistency of the mean ratios of IC density estimates derived from the method involving the manual selection of “three representative images” by the observers. The precision of the IC density estimates was compared to a scenario involving three randomly selected images.ResultsA total of 12 randomly selected, non-overlapping IVCM images were found to be required to produce a corneal epithelial IC density estimate that was within 30% of the true mean, 95% of the time, for the central cornea; seven such images produced an equivalent level of precision in the peripheral cornea. Mean ratios of corneal IC density estimates derived from “three representative images” methods had poor consistency between observers (ICC estimates <0.5) and similar levels of precision when compared with using three randomly selected images (p > 0.05 for all comparisons), in both the central and peripheral cornea.ConclusionsData presented in this study can inform image selection methods, and the sample size required for a preferred level of accuracy, when quantifying IC densities in the central and peripheral corneal epithelium using IVCM images.

Interadministrator Reliability of a Modified Instrumented Push and Release Test of Reactive Balance.

Traditional assessments of reactive balance require sophisticated instrumentation to ensure objective, highly repeatable paradigms. This instrumentation is clinically impractical. The Push and Release test (P&R) is a well-validated clinical test that examines reactive balance, and the application of wearable inertial measurement units (IMU) enables sensitive and objective assessment of this clinically feasible test. The P&R relies on administrator experience and may be susceptible to interadministration reliability concerns. The purpose of this study was to evaluate the interadministrator reliability of objective outcomes from an instrumented, modified version of the P&R test. Crossover interadministrator design. Twenty healthy adults (20-35y) completed the P&R in 4 directions with 2 different administrators. Measures quantified using IMUs included step latency, step length, and time to stability. Lean angle (LA) at release was used as a measure of administration consistency. The intraclass correlation coefficient (ICC) estimate was used to assess interadministrator reliability in each direction. To determine consistency of LA within and across administrators, we calculated the SDs for each rater by direction and the interadministrator reliability of LA using ICC. Across individual directions, the ICC for agreement between raters ranged from .16 to .39 for step latency, from .52 to .62 for time to stability, and from .48 to .84 for step length. Summary metrics across all 4 directions produced higher ICC values. There was poor to moderate consistency in administration based on LA, but LA did not significantly affect any of the outcomes. The modified P&R yields moderate interadministrator reliability and high validity. Summary metrics over all 4 directions (the maximum step latency, the median time to stability, and the median step length) are likely more reliable than direction-specific scores. Variations in body size should also be considered when comparing populations.

Validity and Reliability of Typhoid Risk Factors Questionnaire (TRFQ) in Gombe Metropolis, Gombe State, Nigeria

Typhoid poses a significant threat to human health, particularly in Africa and Asia, where the disease continues to be endemic. Typhoid caused approximately 8.4 million disability-adjusted life-years worldwide in 2017, with 116,814 deaths. While vaccinations effectively eradicate the endemic infection, limited resources frequently constrained vaccination strategies. Thus, an understanding of the risks associated with typhoid would aid in implementing targeted interventions. In many states of Nigeria, including Gombe, data on typhoid risk factors are unavailable. This study aimed to develop and validate a questionnaire on typhoid risk factors in the Gombe Metropolis. The questionnaire items were based on previously published questionnaires on typhoid and localized typhoid risk factors. After obtaining ethical approval, the drafted instrument was pilot tested among a small population and checked for readability. The questionnaire’s content validity was checked using Fleiss’ Multi-rater kappa, while internal consistency and test-retest reliability were checked using Cronbach’s alpha and intraclass correlation coefficient (ICC), respectively. The questionnaire validation results revealed that the instruments’ readability was easy, content validity with Fleiss’ kappa value of 0.623 was good, internal consistency with Cronbach’s alpha value of 0.720 and McDonald’s omega of 0.703, and test-retest reliability with an ICC estimate of 0.736 was good. The questionnaire was valid and reliable for collecting data on typhoid risk factors in Gombe State, Nigeria, to develop targeted health interventions and guide the Government’s resource prioritization and allocation decisions for efficient disease control.

Reliability of histologic assessment for NAFLD and development of an expanded NAFLD activity score.

The NASH Clinical Research Network histologic scoring system, the gold-standard NASH histology assessment for clinical trials, has demonstrated intrarater and interrater variability. An expert panel in a previous systematic Research and Development/University of California Los Angeles (RAND/UCLA) study determined that existing histologic scoring systems do not fully capture NASH disease activity and fibrosis, and standardized definitions of histologic features are needed. We evaluated the reliability of existing and alternate histologic measures and their correlations with a disease activity visual analog scale to propose optimal components for an expanded NAFLD activity score (NAS). Four liver pathologists who were involved in the prior RAND/UCLA study underwent standardized training and multiple discussions with the goal of improving agreement. They were blinded to clinical information and scored histologic measures twice, ≥2 weeks apart, for 40 liver biopsies representing the full spectrum of NAFLD. Index intraclass correlation coefficient (ICC) estimates demonstrated intrarater (0.80-0.85) and interrater (0.60-0.72) reliability. Hepatocyte ballooning items had similar interrater ICCs (0.68-0.79), including those extending scores from 0-2 to 0-4. Steatosis measures (interrater ICCs, 0.72-0.80) correlated poorly with disease activity. Correlations with disease activity were largest for hepatocyte ballooning and Mallory-Denk bodies (MDBs), with both used to develop the expanded NAS (intrarater ICC, 0.90; interrater ICC, 0.80). Fibrosis measures had ICCs of 0.70-0.87. After extensive preparation among a group of experienced pathologists, we demonstrated improved reliability of multiple existing histologic NAFLD indices and fibrosis staging systems. Hepatocyte ballooning and MDBs most strongly correlated with disease activity and were used for the expanded NAS. Further validation including evaluation of responsiveness is required.

Obtaining sound intraclass correlation and variance estimates in three-level models: The role of sampling-strategies

Three-level clustered data commonly occur in social and behavioral research and are prominently analyzed using multilevel modeling. The influence of the clustering on estimation results is assessed with the intraclass correlation coefficients (ICCs), which indicate the fraction of variance in the outcome located at each higher level. However, ICCs are prone to bias due to high requirements regarding the overall sample size and the sample size at each data level. In Monte Carlo simulations, we investigate how these sample characteristics influence the bias of the ICCs and statistical power of the variance components using robust ML-estimation. Results reveal considerable underestimation on Level-3 and the importance of the Level-3 sample size in combination with the ICC sizes. Based on our results, we derive concise sampling recommendations and discuss limits to our inferences.

Smartphone Apps for Vaping Cessation: Quality Assessment and Content Analysis.

BackgroundAs the prevalence of electronic cigarette (e-cigarette) use, or vaping, continues to grow, particularly among young people, so does the need for research and interventions to address vaping.ObjectiveThis study examines the quality of free vaping cessation apps, their contents and features, popularity among users, and adherence to evidence-based principles.MethodsA systematic search of existing apps for vaping cessation was conducted in December 2020. Eligible apps were free, in English, and included features specifically targeting vaping cessation. Each app included in the analysis was used daily for at least seven consecutive days, assessed using the Mobile App Rating Scale, and rated by at least two authors (AK, EL, or SS) based on adherence to evidence-based practices. Intraclass correlation coefficient (ICC) estimates were computed to assess interrater reliability (excellent agreement; ICC 0.92; 95% CI 0.78-0.98).ResultsA total of 8 apps were included in the quality assessment and content analysis: 3 were developed specifically for vaping cessation and 5 focused on smoking cessation while also claiming to address vaping cessation. The mean of app quality total scores was 3.66 out of 5. Existing vaping cessation apps employ similar approaches to smoking cessation apps. However, they are very low in number and have limited features developed specifically for vaping cessation.ConclusionsGiven the lack of vaping cessation interventions at a time when they are urgently needed, smartphone apps are potentially valuable tools. Therefore, it is recommended that these apps apply evidence-based practices and undergo rigorous evaluations that can assess their quality, contents and features, and popularity among users. Through this process, we can improve our understanding of how apps can be effective in helping users quit vaping.

Interrater reliability in the assessment of physiotherapy students

BackgroundReliable and objective assessment of psychomotor skills in physiotherapy students’ education is essential for direct feedback and skill improvement. The aim of this study is to determine the interrater reliability in the assessment process of physiotherapy students and to analyse the assessment behaviour of the examiners.MethodsPhysiotherapy teachers from two different schools assessed students from two different schools performing proprioceptive neuromuscular facilitation (PNF) patterns. An evaluation sheet with a 6-point rating scale and 20 evaluation criteria including an overall rating was used for assessment. The interrater reliability was determined calculating an intraclass-correlation coefficient (ICC) and Krippendorff’s alpha. The assessment behaviour of the examiners was further analysed calculating the location parameters and showing the item response distribution over item in form of a Likert plot.ResultsThe ICC estimates were mostly below 0.4, indicating poor interrater reliability. This was confirmed by Krippendorff’s alpha. The examiners showed a certain central tendency and intergroup bias.Discussion and conclusionThe interrater reliability in this assessment format was rather low. No difference between the two physiotherapy schools concerning the interrater reliability could be identified. Despite certain limitations of this study, there is a definite need for improvement of the assessment process in physiotherapy education to provide the students with reliable and objective feedback and ensure a certain level of professional competence in the students.Trial registrationThe study was approved by the ethics committee of the Medical Faculty RWTH Aachen University (EK 340/16).

The reliability and validity of mobalytics proving ground as a perceptual-motor skill assessment for esports

This study aimed to investigate the test-retest reliability and discriminant validity of the Mobalytics Proving Ground™ assessment for League of Legends. Forty participants (age: 24.15 ± 3.68 y, sex: male = 31, female = 9) were a priori classified into two expertise groups: (1) esports players (age: 22.98 ± 3.64 y, sex: male = 18, female = 2), and (2) controls (age: 25.31 ± 3.42 y, sex: male = 13, female = 7). Participants completed three separate trials (60 s each) online. To assess test-retest reliability, variables displaying normal distributions were analysed using intraclass correlation coefficient (ICC) estimates for two-way mixed-effects models with 95% confidence intervals. The average ICC for all the independent variables in the esports group and control group were moderate (ICC esports = 0.53 and ICC control = 0.72). The average 95% confidence intervals for the independent variables in the esports and control group were ICC = 0.30–0.75 and ICC = 0.55–0.86, respectively. A Friedman test revealed an effect size of 0.11 in the esports group and 0.07 in the control group. In terms of discriminant validity, there were significant differences for 17 variables when comparing the best scores of each group. Overall, the Mobalytics Proving Ground™ assessment used in the current study can, to some extent, distinguish esports players from controls.

A Multi-Laboratory Comparison of Methods for Detection and Quantification of African Swine Fever Virus.

African swine fever is a viral disease of the family Suidae. Methods to detect and quantify African swine fever virus (ASFV) include qPCR and virus infectivity assays. Individual laboratories often use in-house procedures for these assays, which can hamper the comparison of results. The objective of this study was to estimate the probability of ASFV detection using these assays, and to determine the inter-test correlations between results. This was achieved by testing a panel of 80 samples at three reference laboratories. Samples were analysed using nucleic acid extraction and qPCR, as well as virus infectivity assays. For qPCR, a very high probability (ranging from 0.96 to 1.0) of detecting ASFV DNA was observed for all tested systems. For virus infectivity assays in cells, the probability of detecting infectious ASFV varied from 0.68 to 0.90 and was highest using pulmonary alveolar macrophages, followed by MARC145 cells, peripheral blood monocytes, and finally wild boar lung cells. Intraclass correlation coefficient estimates of 0.97 (0.96–0.98) between qPCR methods, 0.80 (0.74–0.85) to 0.94 (0.92–0.96) between virus infectivity assays, and 0.77 (0.68–0.83) to 0.95 (0.93–0.96) between qPCR methods and virus infectivity assays were obtained. These findings show that qPCR gives the highest probability for the detection of ASFV.

Estimating heritability using family-pooled phenotypic and genotypic data: a simulation study applied to aquaculture

Estimating heritability based on individual phenotypic and genotypic measurements can be expensive and labour-intensive in commercial aquaculture breeding. Here, the feasibility of estimating heritability using within-family means of phenotypes and allelic frequencies was investigated. Different numbers of full-sib families and family sizes across ten generations with phenotypic and genotypic information on 10 K SNPs were analysed in ten replicates. Three scenarios, representing differing numbers of pools per family (one, two and five) were considered. The results showed that using one pool per family did not reliably estimate the heritability of family means. Using simulation parameters appropriate for aquaculture, at least 200 families of 60 progeny per family divided equally in two pools per family was required to estimate the heritability of family means effectively. Although application of five pools generated more within- and between- family relationships, it reduced the number of individuals per pool and increased within-family residual variation, hence, decreased the heritability of family means. Moreover, increasing the size of pools resulted in increasing the heritability of family means towards one. In addition, heritability of family mean estimates were higher than family heritabilities obtained from Falconer’s formula due to lower intraclass correlation estimate compared to the coefficient of relationship.

Validation of a Musculoskeletal Digital Assessment Routing Tool: Protocol for a Pilot Randomized Crossover Noninferiority Trial.

BackgroundMusculoskeletal conditions account for 16% of global disability, resulting in a negative effect on millions of patients and an increasing demand for health care use. Digital technologies to improve health care outcomes and efficiency are considered a priority; however, innovations are rarely tested with sufficient rigor in clinical trials, which is the gold standard for clinical proof of safety and efficacy. We have developed a new musculoskeletal digital assessment routing tool (DART) that allows users to self-assess and be directed to the right care. DART requires validation in a real-world setting before implementation.ObjectiveThis pilot study aims to assess the feasibility of a future trial by exploring the key aspects of trial methodology, assessing the procedures, and collecting exploratory data to inform the design of a definitive randomized crossover noninferiority trial to assess DART safety and effectiveness.MethodsWe will collect data from 76 adults with a musculoskeletal condition presenting to general practitioners within a National Health Service (NHS) in England. Participants will complete both a DART assessment and a physiotherapist-led triage, with the order determined by randomization. The primary analysis will involve an absolute agreement intraclass correlation (A,1) estimate with 95% CI between DART and the clinician for assessment outcomes signposting to condition management pathways. Data will be collected to allow the analysis of participant recruitment and retention, randomization, allocation concealment, blinding, data collection process, and bias. In addition, the impact of trial burden and potential barriers to intervention delivery will be considered. The DART user satisfaction will be measured using the system usability scale.ResultsA UK NHS ethics submission was done during June 2021 and is pending approval; recruitment will commence in early 2022, with data collection anticipated to last for 3 months. The results will be reported in a follow-up paper in 2022.ConclusionsThis study will inform the design of a randomized controlled crossover noninferiority study that will provide evidence concerning mobile health DART system clinical signposting in an NHS setting before real-world implementation. Success should produce evidence of a safe, effective system with good usability, potentially facilitating quicker and easier patient access to appropriate care while reducing the burden on primary and secondary care musculoskeletal services. This rigorous approach to mobile health system testing could be used as a guide for other developers of similar applications.Trial RegistrationClinicalTrials.gov NCT04904029; http://clinicaltrials.gov/ct2/show/NCT04904029International Registered Report Identifier (IRRID)PRR1-10.2196/31541