Poor Intraclass Correlation Research Articles

Characterization of renal masses with MRI-based radiomics: assessment of inter-package and inter-observer reproducibility in a prospective pilot study.

To evaluate radiomics features' reproducibility using inter-package/inter-observer measurement analysis in renal masses (RMs) based on MRI and to employ machine learning (ML) models for RM characterization. 32 Patients (23M/9F; age 61.8 ± 10.6years) with RMs (25 renal cell carcinomas (RCC)/7 benign masses; mean size, 3.43 ± 1.73cm) undergoing resection were prospectively recruited. All patients underwent 1.5T MRI with T2-weighted (T2-WI), diffusion-weighted (DWI)/apparent diffusion coefficient (ADC), and pre-/post-contrast-enhanced T1-weighted imaging (T1-WI). RMs were manually segmented using volume of interest (VOI) on T2-WI, DWI/ADC, and T1-WI pre-/post-contrast imaging (1-min, 3-min post-injection) by two independent observers using two radiomics software packages for inter-package and inter-observer assessments of shape/histogram/texture features common to both packages (104 features; n = 26 patients). Intra-class correlation coefficients (ICCs) were calculated to assess inter-observer and inter-package reproducibility of radiomics measurements [good (ICC ≥ 0.8)/moderate (ICC = 0.5-0.8)/poor (ICC < 0.5)]. ML models were employed using reproducible features (between observers and packages, ICC > 0.8) to distinguish RCC from benign RM. Inter-package comparisons demonstrated that radiomics features from T1-WI-post-contrast had the highest proportion of good/moderate ICCs (54.8-58.6% for T1-WI-1min), while most features extracted from T2-WI, T1-WI-pre-contrast, and ADC exhibited poor ICCs. Inter-observer comparisons found that radiomics measurements from T1-WI pre/post-contrast and T2-WI had the greatest proportion of features with good/moderate ICCs (95.3-99.1% T1-WI-post-contrast 1-min), while ADC measurements yielded mostly poor ICCs. ML models generated an AUC of 0.71 [95% confidence interval = 0.67-0.75] for diagnosis of RCC vs. benign RM. Radiomics features extracted from T1-WI-post-contrast demonstrated greater inter-package and inter-observer reproducibility compared to ADC, with fair accuracy for distinguishing RCC from benign RM. Knowledge of reproducibility of MRI radiomics features obtained on renal masses will aid in future study design and may enhance the diagnostic utility of radiomics models for renal mass characterization.

The test–retest reliability of physiological and perceptual responses during treadmill load carriage

PurposeUnderstanding the test–retest reliability of physiological responses to load carriage influences the interpretation of those results. The aim of this study was to determine the test–retest reliability of physiological measures during loaded treadmill walking at 5.5 km h−1 using the MetaMax 3B.MethodsFifteen Australian Army soldiers (9 male, 6 female) repeated two 12-min bouts of treadmill walking at 5.5 km h−1 in both a 7.2 kg Control condition (MetaMax 3B, replica rifle) and a 23.2 kg Patrol condition (Control condition plus vest) across three sessions, separated by one week. Expired respiratory gases and heart rate were continuously collected, with the final 3 min of data analysed. Ratings of Perceived Exertion and Omnibus-Resistance Exercise Scale were taken following each trial. Reliability was quantified by coefficient of variation (CV), intra-class correlation coefficients (ICC), smallest worthwhile change (SWC), and standard error of the measurement.ResultsMetabolic and cardiovascular variables were highly reliable (≤ 5% CV; excellent-moderate ICC), while the respiratory variables demonstrated moderate reliability (< 8% CV; good-moderate ICC) across both conditions. Perceptual ratings had poorer reliability during the Control condition (12–45% CV; poor ICC) than the Patrol condition (7–16% CV; good ICC).ConclusionsThe test–retest reliability of metabolic and cardiovascular variables was high and relatively consistent during load carriage. Respiratory responses demonstrated moderate test–retest reliability; however, as the SWC differed with load carriage tasks, such data should be interpreted independently across loads. Perceptual measures demonstrated poor to moderate reliability during load carriage, and it is recommended that they only be employed as secondary measures.Graphical abstract

Episiotomy: Are Indian Obstetricians Getting the Angle Right?

BackgroundA mediolateral episiotomy is recommended when indicated at a 60° angle at crowning, to avoid obstetric anal sphincter injuries (OASIs) by episiotomies angled too close or distant to the anus. This study surveyed obstetricians in India regarding the recommended episiotomy angle and their ability to correctly draw the angle.MethodsWorkshops were conducted in India to share knowledge in the prevention and repair of OASIs. A questionnaire was distributed prior to the workshop. Participants were asked to describe the recommended episiotomy angle and to draw this on a paper replica of the perineum. The intra-class correlation coefficient (ICC) was calculated to assess the inter-rater reliability between the angle stated and drawn. A 2° difference was deemed acceptable. Standard errors of measurement (SEM) were calculated to measure the range of error of each measurement.ResultsOne hundred and forty doctors participated. 47.9% described the angle of an episiotomy to be 60°. Only 2.2% drew an angle of 60°, but 8.7% (n = 12) drew between 58 and 62°. Only 5.8% (n = 6) of doctors correctly drew the episiotomy angle they described. There was poor agreement ICC = 0.18 (− 0.01 to 0.36) with a SEM of ± 12.2°ConclusionsKnowledge surrounding the recommended episiotomy angle is lacking. Doctors are failing to estimate their desired episiotomy angle. This highlights the need for national guidelines, the creation and validation of structured training programmes to improve accuracy, or using fixed-angle devices such as the EPISCISSORS-60 or other proven measurement aids to minimise preventable harm due to human error.

Chirp-Evoked VEMPs: A Test-Retest Reliability Study.

To determine the test-retest reliability of cervical and ocular vestibular-evoked myogenic potentials (c&oVEMP) evoked by 500 Hz narrowband (NB) CE-Chirp and broadband (BB) CE-Chirp stimuli. Twenty healthy participants (10 female) were tested twice on the same day to determine the within-session reliability and 1 week later to determine the between-session reliability. The latency, amplitude, and asymmetry ratio of c&oVEMPs elicited by 95 dB nHL air-conducted (AC) 500 Hz NB CE-Chirp and BB CE-Chirp were recorded bilaterally. A moderate to good between-session reliability with intraclass correlation coefficient (ICC) values ranging from 0.52 to 0.82 was observed for cVEMP latency, amplitude, and asymmetry ratio evoked by 500 Hz NB CE-Chirp, as well as for the BB CE-Chirp cVEMP amplitude (ICC of 0.70 and 0.84). In contrast, an overall poor reliability ICC values between 0.30 and 0.42 for cVEMP latency and asymmetry ratio were observed for BB CE-Chirp. For the oVEMP, overall poor between-session reliability for all response parameters evoked by the 500 Hz NB CE-Chirp and the BB CE-Chirp was observed. The 500 Hz NB CE-Chirp was more reliable than the BB CE-Chirp in terms of cVEMP latency, amplitude, and asymmetry ratio. Further investigation using the standard electrode montage is necessary to assess the test-retest reliability of the chirp-evoked oVEMP.

Automated Radiographic Measurements of Knee Osteoarthritis.

Herewith, we report the development of Orthopedic Digital Image Analysis (ODIA) software that is developed to obtain quantitative measurements of knee osteoarthritis (OA) radiographs automatically. Manual segmentation and measurement of OA parameters currently hamper large-cohort analyses, and therefore, automated and reproducible methods are a valuable addition in OA research. This study aims to test the automated ODIA measurements and compare them with available manual Knee Imaging Digital Analysis (KIDA) measurements as comparison. This study included data from the CHECK (Cohort Hip and Cohort Knee) initiative, a prospective multicentre cohort study in the Netherlands with 1,002 participants. Knee radiographs obtained at baseline of the CHECK cohort were included and mean medial/lateral joint space width (JSW), minimal JSW, joint line convergence angle (JLCA), eminence heights, and subchondral bone intensities were compared between ODIA and KIDA. Of the potential 2,004 radiographs, 1,743 were included for analyses. Poor intraclass correlation coefficients (ICCs) were reported for the JLCA (0.422) and minimal JSW (0.299). The mean medial and lateral JSW, eminence height, and subchondral bone intensities reported a moderate to good ICC (0.7 or higher). Discrepancies in JLCA and minimal JSW between the 2 methods were mostly a problem in the lateral tibia plateau. The current ODIA tool provides important measurements of OA parameters in an automated manner from standard radiographs of the knee. Given the automated and computerized methodology that has very high reproducibility, ODIA is suitable for large epidemiological cohorts with various follow-up time points to investigate structural progression, such as CHECK or the Osteoarthritis Initiative (OAI).

Reliability of electroencephalogram indicator and event related potential in subacute stroke

Cognitive impairment is observed in 12% to 56% of stroke patients, and screening for cognitive impairment is often complex and time-consuming, with results dependent on patient compliance. Therefore, there is a need for an objective method to assess cognitive impairment regardless of patient compliance. Objective evaluation methods include electroencephalogram (EEG) and event-related potential (ERP). This study was conducted to assess intra-tester reliability of resting EEG-based spectral features and auditory/visual P300 latency/amplitude in patients with subacute ischemic stroke. Twenty patients with subacute ischemic stroke were included in the study. The resting EEG and P300 wave using an auditory and visual oddball paradigm were measured at baseline and once again in 24 hours. The following electrode positions (10-20 system) were constantly recorded: F3 (Frontal), Fz, F4, C3 (Central), Cz, C4, P3 (Parietal), Pz, P4. DAR (delta/alpha ratio) and BSI (brain symmetry index) were determined using EEG data. F3 and F4, C3 and C4 and P3 and P4 were switched according to the stroke side and classified as affected hemisphere (AH) and unaffected hemisphere (UH) after the evaluation. In ERP, the amplitude and latency of P300 were obtained. In reliability analysis of EEG-based spectral characteristics, significant reliability was observed for DAR (ICC = 0.447), BSldir (ICC = 0.713) and BSIdirtheta (ICC = 0.724) (Table 4). DAR was showed a poor ICC level, and BSIdir and BSIdirtheta had a moderate ICC level. Visual P300 latency showed excellent intraclass correlation coefficient (ICC) in several montages (PUH = 0.972, Pz = 0.945). In 6 montages, auditory P300 latency was reliable, while in 9 montages, visual P300 latency was reliable. In 4 montages, auditory P300 amplitude was reliable, while visual P300 amplitude was reliable in 7. The visual P300 was more reliable than the auditory P300. The ICC values for P300 latency were greater than those for amplitude. Therefore, when ERP is performed on subacute stroke patients, visual has higher reliability than auditory.

Interobserver Variability in Assessment of Depth of Submucosal Invasion for Colonic Endoscopic Resections Among Subspecialized Gastrointestinal Pathologists.

Recent data support that low-risk submucosally invasive (pT1) colonic adenocarcinomas (ie, completely resected tumors that lack high-grade morphology, tumor budding, and lymphovascular invasion) are considered cured via endoscopic resection, provided that the submucosal invasion is less than 1000 μm. Hence, the pathologists' assessment of depth of submucosal invasion may guide further management (ie, surveillance versus colectomy). To assess interobserver concordance among gastrointestinal pathologists in measuring submucosal depth of invasion in colonic endoscopic resections. Six gastrointestinal pathologists from 5 academic centers independently measured the greatest depth of submucosal invasion in micrometers on 52 hematoxylin-eosin-stained slides from colonic endoscopic specimens with pT1 adenocarcinomas, per published guidelines (round 1 scoring). Two separate measurements (round 2 scoring) were subsequently performed by each pathologist following a consensus meeting, (1) from surface of lesion and (2) from muscularis mucosae, and pathologists were asked to choose their (3) "real-life (best)" assessment between the first 2 measurements. Interobserver agreement was assessed by the intraclass correlation coefficient (ICC) and Cohen κ statistics. Round 1 had poor ICC (0.43; 95% CI, 0.31-0.56). Round 2 agreement was good when measuring from surface (ICC = 0.83; 95% CI, 0.76-0.88) but moderate (ICC = 0.59; 95% CI, 0.47- 0.70) when measuring from muscularis mucosae and became poor (ICC = 0.49; 95% CI, 0.36-0.61) for the best-assessment measurement. Our findings indicate that clearer and reproducible guidelines are needed if clinical colleagues are to base important management decisions on pathologists' estimate of depth of submucosal invasion in the colonic endoscopic resections.

Measuring depth of invasion of submucosa - invasive adenocarcinoma in oesophageal endoscopic specimens: how good are we?

Emerging data support that submucosa-invasive (pT1b) esophageal adenocarcinomas are cured via endoscopic resection, provided that invasion measures ≤500 μm, they lack other histological features predictive of nodal metastasis and have negative margins. Hence, pathologists' measurement of the depth of submucosal invasion in endoscopic resections may dictate further management (i.e. endoscopic follow-up versus oesophagectomy). In this study, we assessed the interobserver agreement in measuring the depth of submucosal invasion in oesophageal endoscopic resections. Six subspecialised gastrointestinal (GI) pathologists from five academic centres independently measured the depth of submucosal invasion in μm from the deepest muscularis mucosae on 37 oesophageal endoscopic resection slides (round 1 scoring). A consensus meeting with a systematic approach for measuring and discussion of pitfalls was undertaken and remeasuring (round 2 scoring) was conducted. Interobserver agreement was assessed by the intraclass correlation coefficient (ICC) and Cohen's kappa statistics. A lack of agreement was seen among the six reviewers with a poor ICC for both rounds: 1 [0.40, 95% confidence interval (CI) = 0.26-0.56] and 2 (0.49, 95% CI = 0.34-0.63). When measurements were categorised as < or >500 μm, the overall agreement among the six reviewers was only fair for both rounds: 1 (kappa = 0.37, 95% CI = 0.22-0.53) and 2 (kappa = 0.29, 95% CI = 0.12-0.46). Our study shows a lack of agreement among gastrointestinal pathologists in measuring the depth of submucosal invasion in oesophageal endoscopic resections despite formulating a consensus approach for scoring. If important management decisions continue to be based upon this parameter, more reproducible and concrete guidelines are needed.

Test Re-test Reliability of Dual-site TMS Measures of SMA-M1 Connectivity Differs Across Inter-stimulus Intervals in Younger and Older Adults

Dual-site transcranial magnetic stimulation (TMS) is a promising tool to measure supplementary motor area and primary motor cortex (SMA-M1) connectivity in younger and older adults, and could be used to understand the pathophysiology of movement disorders. However, test re-test reliability of dual-site TMS measures of SMA-M1 connectivity has not been established. We examined the reliability of SMA-M1 connectivity using dual-site TMS in two sessions in 30 younger and 30 older adults. For dual-site TMS, a conditioning pulse delivered to SMA (140% of active motor threshold) preceded a test pulse delivered to M1 (intensity that elicited MEPs of ~1 mV) by inter-stimulus intervals (ISI) of 6 ms, 7 ms, and 8 ms. Moderate intraclass correlation coefficients (ICC) were found for SMA-M1 connectivity at an ISI of 7 ms in younger (ICC: 0.69) and older adults (ICC: 0.68). Poor ICCs were found for SMA-M1 connectivity at ISIs of 6 ms and 8 ms in both age groups (ICC range: 0.01–0.40). We report evidence for stable measures of SMA-M1 connectivity at an ISI of 7 ms in both age groups. These findings are foundational for future research developing evidence-based interventions to strengthen SMA-M1 connectivity to improve bilateral motor control in older adults and populations with movement disorders.

The reporters’ agreement in assessing the quality of life of young people with intellectual disabilities

A good evaluation of the quality of life (QoL) of young people with disability is essential to detect, monitor and report their support needs in research, and individual, institutional and policy planning. The aim of this study is to evaluate the reporters’ agreement in the assessment of the QoL of young people with Intellectual Disabilities in transition to adulthood. Self and proxy-report version of INICO-FEAPS quality of life scale was used to assess 119 students of a post-compulsory educational program, being the proxy-report informed by one of their relatives. Results show poor intraclass correlations between informants. No discrepancies were found in the global QoL. However, small discrepancies were found for some dimensions. Young people’s reports were higher for self-determination, personal development and interpersonal relationships. On the contrary, relatives’ reports were higher for rights, material and physical wellbeing. The discrepancies in global QoL assessment were not related to student’s severity, IQ, diagnosis or parent’s gender. These findings underline the importance of young people’s participation in the assessment of their QoL as well as the convenience of using both kind of informants when taking appropriate decisions in educational contexts during transition to adulthood.

Prognostic value of texture analysis from cardiac magnetic resonance imaging in patients with Takotsubo syndrome: a machine learning based proof-of-principle approach

Cardiac magnetic resonance (CMR) imaging has become an important technique for non-invasive diagnosis of takotsubo syndrome (TTS). The long-term prognostic value of CMR imaging in TTS has not been fully elucidated yet. This study sought to evaluate the prognostic value of texture analysis (TA) based on CMR images in patients with TTS using machine learning. In this multicenter study (InterTAK Registry), we investigated CMR imaging data of 58 patients (56 women, mean age 68 ± 12 years) with TTS. CMR imaging was performed in the acute to subacute phase (median time after symptom onset 4 days) of TTS. TA of the left ventricle was performed using free-hand regions-of-interest in short axis late gadolinium-enhanced and on T2-weighted (T2w) images. A total of 608 TA features adding the parameters age, gender, and body mass index were included. Dimension reduction was performed removing TA features with poor intra-class correlation coefficients (ICC ≤ 0.6) and those being redundant (correlation matrix with Pearson correlation coefficient r > 0.8). Five common machine-learning classifiers (artificial neural network Multilayer Perceptron, decision tree J48, NaïveBayes, RandomForest, and Sequential Minimal Optimization) with tenfold cross-validation were applied to assess 5-year outcome including major adverse cardiac and cerebrovascular events (MACCE). Dimension reduction yielded 10 TA features carrying prognostic information, which were all based on T2w images. The NaïveBayes machine learning classifier showed overall best performance with a sensitivity of 82.9% (confidence interval (CI) 80–86.2), specificity of 83.7% (CI 75.7–92), and an area-under-the receiver operating characteristics curve of 0.88 (CI 0.83–0.92). This proof-of-principle study is the first to identify unique T2w-derived TA features that predict long-term outcome in patients with TTS. These features might serve as imaging prognostic biomarkers in TTS patients.

Effects of cognitive and motor dual-task on spatiotemporal gait parameters in children and adolescents with Cerebral Palsy

Single-task (ST) and dual-task (DT) assessments are commonly used to evaluate motor and cognitive impairment in people with multiple sclerosis (MS). Although variability can influence repeated DT testing measures, the reliability of several DT variables over time has not been adequately explored. For instance, a third testing session has never been included to observe whether DT has a learning effect. DT cognition rate reliability has not yet been examined and dual-task cost (DTC), a widely used calculation for DT interpretation, has not been proven reliable.To evaluate the reliability of ST and DT measures of gait and cognition over three test sessions.This was a cross-sectional study involving 18 people with MS and 12 controls. Participants attended three test sessions, each one week apart. ST and DT (serial seven subtraction) gait variables, DTC, coefficient of variability (CV), and cognition rate were extracted and calculated using an instrumented walkway. Reliability was assessed using intraclass correlation coefficients (ICC) or Kendall’s coefficient of concordance (KCC; nonparametric test) and minimum detectable change (MDC); between-session learning effect was assessed using repeated measures ANOVA.ICC/KCC values for ST and DT gait variables ranged from moderate to excellent (0.50-0.99). However, reliability for DT stride width and cognition rate was lower in controls. In general, DTC and CV variables had poor ICCs and high MDC values (49.19–1478.67 %), although some DTC variables had moderate or higher reliability in controls. Cognition rate was reliable in both MS (ICC 0.91) and controls (ICC 0.84). A learning effect between sessions was observed for DT velocity in both groups and for DTC cadence in people with MS.ST and DT gait measures as well as DT cognition rate are reliable outcomes for repeated testing, while DTC and CV variables may not be suitable for long-term monitoring.

The reliability of surface EMG derived motor unit variables

Motor unit (MU) recordings obtained from surface electromyography (sEMG) decomposition are used to investigate the neural control of muscle in response to interventions, but our understanding of the longer-term reliability of MU variables is limited. This study examined the reliability of MU variables in the flexor carpi radialis (FCR) and tibialis anterior (TA) over a three-month period. Forty college-aged participants completed isometric wrist flexion (n = 20) and dorsiflexion (n = 20). There were 3 maximal isometric voluntary contractions (MVC) and 3 ramp contractions to 60% of MVC on four separate sessions separated by a total of 13 weeks. Intraclass correlation coefficients (ICC) were calculated from a fully nested ANOVA model. Maximal force was highly reliable (ICC = 0.94–0.99). The ICC values ranged from 0.49 to 0.92 for the FCR MU variables and from 0.58 to 0.96 for the TA MU variables. All MU variables exhibited a high degree of stability of means across test session and consistency within subjects, with the exception of the number of MUs detected in the TA. Poor ICC values did not reflect poor reliability but rather, convergence towards a narrow range of physiologically normal values. Surface EMG decomposition of a large population of MUs showed no differences in common drive between FCR (0.273) and for the TA (0.267) across test sessions. Forty percent of the sampled MUs in both muscles had a common drive of 0.30 or greater, which provides indirect support for the validity of the decompositions. MU variables may be used to monitor adaptations to a longer-term intervention study.

Image Distortion in Biplanar Slot Scanning: Patient-specific Factors.

Error within imaging measurements can be due to processing, magnification, measurement performance, or patient-specific factors. Previous length measurement studies based on radiographs have shown good intraclass correlation coefficients (ICCs) on single images; but have not assessed interimage distortion. In our study, "image distortion in biplanar slot scanning: technology-specific factors" we determined that there is minimal image distortion due to the image acquisition when using biplanar slot scanning. In this study, we aim to determine the role of patient-specific factors in image distortion, specifically evaluating interimage distortion. Digital radiographs and biplanar slot scanner images were reviewed in 43 magnetically controlled growing rod (MCGR) patients. Fifty-five postoperative anteroposterior digital radiographs, 184 follow-up biplanar slot-scanner scanner posteroanterior and 76 biplanar slot-scanner scanner laterals were measured by 2 residents and 1 attending. The manufacturer reported average actuator diameter of 9.02 mm was used as our reference width. Overall, within image interobserver ICC were moderate to excellent (0.635 to 0.983), but the interimage ICCs were poor (0.332). Digital radiographs consistently overestimated the MCGR actuator width (mean=9.655) and biplanar slot-scanner scanner images underestimated it (mean=8.935). The measurement range was large with biplanar slot-scanner scanner posteroanterior (up to 15%) and lateral (22%) measurements and with digital radiographs (39%). Patients with abnormal muscle tone had higher degrees of measurement variability. We found that neither biplanar slot scanning nor digital radiography was precise or accurate. Digital radiographs consistently overestimated MCGR actuator width and biplanar slot scanning underestimated it. The poor ICC's within and between image subtypes and large standard error of measurement reflected a magnitude of distortion that needs to be accounted for when using length measurements clinically. Unlike the clinically insignificant error that we noted in our previous study "image distortion in biplanar slot scanning: technology-specific factors" (0.5% to 1.5% of the measurement), the error noted in this study (0.2% to 38.5% of the measurement) has the potential to be clinically significant. Patients who have abnormal muscle tone had larger measurement errors, likely stemming from motion during the slot scanning process. Level III.

Reliability of gait and dual-task measures in multiple sclerosis

BackgroundSingle-task (ST) and dual-task (DT) assessments are commonly used to evaluate motor and cognitive impairment in people with multiple sclerosis (MS). Although variability can influence repeated DT testing measures, the reliability of several DT variables over time has not been adequately explored. For instance, a third testing session has never been included to observe whether DT has a learning effect. DT cognition rate reliability has not yet been examined and dual-task cost (DTC), a widely used calculation for DT interpretation, has not been proven reliable. Research questionTo evaluate the reliability of ST and DT measures of gait and cognition over three test sessions. MethodsThis was a cross-sectional study involving 18 people with MS and 12 controls. Participants attended three test sessions, each one week apart. ST and DT (serial seven subtraction) gait variables, DTC, coefficient of variability (CV), and cognition rate were extracted and calculated using an instrumented walkway. Reliability was assessed using intraclass correlation coefficients (ICC) or Kendall’s coefficient of concordance (KCC; nonparametric test) and minimum detectable change (MDC); between-session learning effect was assessed using repeated measures ANOVA. ResultsICC/KCC values for ST and DT gait variables ranged from moderate to excellent (0.50-0.99). However, reliability for DT stride width and cognition rate was lower in controls. In general, DTC and CV variables had poor ICCs and high MDC values (49.19–1478.67 %), although some DTC variables had moderate or higher reliability in controls. Cognition rate was reliable in both MS (ICC 0.91) and controls (ICC 0.84). A learning effect between sessions was observed for DT velocity in both groups and for DTC cadence in people with MS. SignificanceST and DT gait measures as well as DT cognition rate are reliable outcomes for repeated testing, while DTC and CV variables may not be suitable for long-term monitoring.

Patient-related factors may influence nursing perception of sleep in the Intensive Care Unit.

ObjectiveThere exist conflicting data regarding the accuracy of ICU nurses accurately assessing patient sleep using validated questionnaires. Using the Richards-Campbell Sleep Questionnaire (RSQ), we hypothesize that patient factors might influence nursing perception of their sleep.MethodsPatients in the ICU who met the inclusion criteria were asked to complete the sleep questionnaire, as were their nurses and intraclass correlation analysis was done.Results38 paired patient-nurse surveys were included for analysis. The mean difference in total average score of the RSQ was not significantly different between patients and nurses. There was fair intraclass correlation by patient age, black race, and admission for respiratory illnesses. A good intraclass correlation existed for non-blacks and admission for non-respiratory reasons. Most striking was the intraclass correlation by sex, with poor intraclass correlation for women compared to an excellent correlation for men.ConclusionThe results of our study confirm that patients in our ICU have poor sleep with a fair intraclass correlation. When examined by patient related factor, the greatest divergence between patient and nursing perception of sleep in the ICU using the RCSQ was patient female sex. More research is needed in this area to better understand the divergence and improve sleep in the ICU.

Validity Evaluation of the Fitbit Charge2 and the Garmin vivosmart HR+ in Free-Living Environments in an Older Adult Cohort.

BackgroundFew studies have investigated the validity of mainstream wrist-based activity trackers in healthy older adults in real life, as opposed to laboratory settings.ObjectiveThis study explored the performance of two wrist-worn trackers (Fitbit Charge 2 and Garmin vivosmart HR+) in estimating steps, energy expenditure, moderate-to-vigorous physical activity (MVPA) levels, and sleep parameters (total sleep time [TST] and wake after sleep onset [WASO]) against gold-standard technologies in a cohort of healthy older adults in a free-living environment.MethodsOverall, 20 participants (>65 years) took part in the study. The devices were worn by the participants for 24 hours, and the results were compared against validated technology (ActiGraph and New-Lifestyles NL-2000i). Mean error, mean percentage error (MPE), mean absolute percentage error (MAPE), intraclass correlation (ICC), and Bland-Altman plots were computed for all the parameters considered.ResultsFor step counting, all trackers were highly correlated with one another (ICCs>0.89). Although the Fitbit tended to overcount steps (MPE=12.36%), the Garmin and ActiGraph undercounted (MPE 9.36% and 11.53%, respectively). The Garmin had poor ICC values when energy expenditure was compared against the criterion. The Fitbit had moderate-to-good ICCs in comparison to the other activity trackers, and showed the best results (MAPE=12.25%), although it underestimated calories burned. For MVPA levels estimation, the wristband trackers were highly correlated (ICC=0.96); however, they were moderately correlated against the criterion and they overestimated MVPA activity minutes. For the sleep parameters, the ICCs were poor for all cases, except when comparing the Fitbit with the criterion, which showed moderate agreement. The TST was slightly overestimated with the Fitbit, although it provided good results with an average MAPE equal to 10.13%. Conversely, WASO estimation was poorer and was overestimated by the Fitbit but underestimated by the Garmin. Again, the Fitbit was the most accurate, with an average MAPE of 49.7%.ConclusionsThe tested well-known devices could be adopted to estimate steps, energy expenditure, and sleep duration with an acceptable level of accuracy in the population of interest, although clinicians should be cautious in considering other parameters for clinical and research purposes.

Clinical and analytical validation of Ki-67 in 9069 patients from IBCSG VIII + IX, BIG1-98 and GeparTrio trial: systematic modulation of interobserver variance in a comprehensive in silico ring trial.

Ki-67 has been clinically validated for risk assessment in breast cancer, but the analytical validation and cutpoint-definition remain a challenge. Intraclass correlation coefficients (ICCs) are a statistical parameter for Ki-67 interobserver performance. However, the maximum degree of variance among pathologists allowed for meaningful biomarker results has not been defined. Different amounts of variance were added to central pathology Ki-67 data (n = 9069) from three cohorts (IBCSGVIII + IX, BIG1-98, GeparTrio) by simulation of 4500 evaluations for each cohort, which were grouped by ICCs, ranging from excellent (ICC = 0.9) to poor concordance (ICC = 0.1). Endpoints were disease-free survival (DFS) and pathological complete response (pCR, GeparTrio). Ki-67 was a significant continuous prognostic marker for DFS over a wide range of cutpoints between 8% and 30% in all three cohorts. In our modelling approach, Ki-67 was a stable prognostic marker despite increased interpathologist variance. Even for a poor ICC of 0.5, one or more significant Ki-67 cutoffs were detected in 86.8% (GeparTrio), 92.4% (IBCSGVIII + IX) and 100% of analyses (BIG1-98). Similarly, in GeparTrio, even with an extremely low ICC of 0.2, 99.6% of analyses were significant for pCR. Our study shows that Ki-67 is a continuous marker which is extremely robust to pathologist variation. Even if only 50% of variance is attributable to true Ki-67-based proliferation (ICC = 0.5), this information is sufficient to obtain statistically significant differences in clinical cohorts. This stable performance explains the observation that many Ki-67 studies achieve significant results despite relevant interobserver variance and points to a high clinical validity of this biomarker. For clinical decisions based on analysis of individual patient data, ongoing efforts to further reduce interobserver variability, including ring trials and standardized guidelines as well as image analysis approaches, should be continued.

Day-level sedentary pattern estimates derived from hip-worn accelerometer cut-points in 8–12-year-olds: Do they reflect postural transitions?

ABSTRACTImproving sedentary measurement is critical to understanding sedentary-health associations in youth. This study assessed agreement between the thigh-worn activPAL and commonly used hip-worn ActiGraph accelerometer methods for assessing sedentary patterns in children. Both devices were worn by 8–12-year-olds (N = 195) for 4.6 ± 1.9 days. Two ActiGraph cut-points were applied to two epoch durations: ≤25 counts (c)/15 s, ≤75c/15s, ≤100c/60s, and ≤300c/60s. Bias, mean absolute deviation (MAD), and intraclass correlation coefficients (ICCs) tested agreement between devices for total sedentary time and 11 sedentary pattern variables (usual bout duration, sedentary time accumulated in various bout durations, breaks/day, break rate, and alpha). For most sedentary pattern variables, ActiGraph 25c/15s, 75c/15s, and 100c/60s had poor ICCs, with bias and MAD >20%. ActiGraph 300c/60s had a better agreement than the other cut-points, but all ICCs were <0.587. ActiGraph underestimated sedentary time in longer bouts and usual bout duration, and overestimated sedentary time in shorter bouts, breaks/day, and alpha. For total sedentary time, ActiGraph 25c/15s, 300c/60s, and 75c/15s had good/fair ICCs, with bias and MAD <20%. Sedentary patterns derived from two commonly used ActiGraph cut-points did not appear to reflect postural changes. These differences between measurement devices should be considered when interpreting findings from sedentary pattern studies.

Magnetic Resonance Elastography in Primary Sclerosing Cholangitis: Interobserver Agreement for Liver Stiffness Measurement with Manual and Automated Methods

Primary sclerosing cholangitis, a chronic liver disease causes heterogeneous parenchymal changes and fibrosis. Liver stiffness measurement (LSM) with magnetic resonance Elastography (MRE) may be affected by this heterogeneous distribution. We evaluated interobserver agreement of LSM in primary sclerosing cholangitis (PSC) with manual and automated methods to study the influence of heterogeneous changes. A total of 79 consecutive patients with PSC who had a liver MRI and MRE formed the study group. Three readers with 1-3 years' experience in MRE and a MRE expert (11 years' experience) independently performed LSM. Each reader manually drew free hand (fROI) and average (aROI) on stiffness maps. Automatic liver elasticity calculation (ALEC) was used to generate automated LSM. The expert fROI was the reference standard. Correlation analysis and absolute intra-class correlation coefficient (ICC) analysis was performed. LSM data of 79 livers and 315 sections were evaluated. There was excellent ICC between expert and reader fROIs (0.989, 95% confidence interval, and 0.985-0.993) and aROIs (0.971, 95% confidence interval, and 0.953-0.983) and ALEC (0.972, 0.957-0.982) with fROI performing better. The areas measured with fROIs and ALEC had moderate ICC with Expert fROI (0.64 and 0.56, respectively) whereas aROI area had a poor ICC of 0.12. Comparison of multiple methods showed significant differences in LSM between expert fROI and aROI of two readers and no significant differences for fROIs of all three readers. LSM with MRE in PSC patients shows excellent interobserver agreement with both fROI and aROI methods with better performance with fROI. fROI may therefore be preferred for LSM measurements in PSC.