Reference Standard Assessment Research Articles

Gadolinium Enhanced T2 FLAIR is an Imaging Biomarker of Radiation Necrosis and Tumor Progression in Patients with Brain Metastases.

Differentiating radiation necrosis (RN) from tumor progression (TP) after radiotherapy for brain metastases is an important clinical problem requiring advanced imaging techniques which may not be widely available and are challenging to perform at multiple time points. The ability to leverage conventional MRI for this problem, could have meaningful clinical impact. The purpose of this study was to explore contrast enhanced T2 FLAIR (T2FLAIRc) as a new imaging biomarker of RN and TP. This single-institution retrospective study included patients with treated brain metastases undergoing DSC-MRI between January 2021 and June 2023. Reference standard assessment was based on histopathology or serial follow-up including results of DSC-MRI for a minimum of 6 months from the first DSC-MRI. The index test was implemented as part of the institutional brain tumor MRI protocol and preceded the first DSC-MRI. T2FLAIRc and gadolinium enhanced T1 MPRAGE (T1c) signal were normalized against normal brain parenchyma and expressed as z-score. Mean signal intensity of enhancing disease for RN and TP groups were compared using unpaired t-test. Receiver operator characteristic (ROC) curves and area under the ROC curve (AUC) were derived by bootstrapping. DeLong test was used to compare AUC. 56 participants (mean age, 62 years +/-12.7 [SD]; 39 females); 28 RN, 28 TP were evaluated. The index MRI was performed on average 73 days +/-34 [SD] before the first DSC-MRI. Significantly higher z-scores were found for RN using T2FLAIRc (8.3 versus 5.8, p<0.001) and T1c (4.1 versus 3.5, p=0.02). AUC for T2FLAIRc (0.83, 95% CI, 0.72-0.92) was greater than T1c (0.70, 95% CI, 0.560.83) (p = 0.04). The AUC of DSC derived rCBV (0.82, 95% CI, 0.70-0.93) was not significantly different from T2FLAIRc (p = 0.9). A higher normalized T1c and T2FLAIRc signal intensity was found for RN. In a univariable test, mean T2FLAIRc signal intensity of enhancing voxels showed good discrimination performance for distinguishing RN from TP. The results of this work demonstrate the potential of T2FLAIRc as an imaging biomarker in the work-up of RN in patients with brain metastases. AUC = area under the receiver operating characteristic curve; RN = radiation necrosis; ROC = receiver operating characteristic; SRS = stereotactic radiosurgery; T1c = contrast enhanced T1; T2FLAIRc = contrast enhanced T2 FLAIR; TP = tumor progression.

LI-RADS CT and MRI Ancillary Feature Association with Hepatocellular Carcinoma and Malignancy: An Individual Participant Data Meta-Analysis.

Background The independent contribution of each Liver Imaging Reporting and Data System (LI-RADS) CT or MRI ancillary feature (AF) has not been established. Purpose To evaluate the association of LI-RADS AFs with hepatocellular carcinoma (HCC) and malignancy while adjusting for LI-RADS major features through an individual participant data (IPD) meta-analysis. Materials and Methods Medline, Embase, Cochrane Central Register of Controlled Trials, and Scopus were searched from January 2014 to January 2022 for studies evaluating the diagnostic accuracy of CT and MRI for HCC using LI-RADS version 2014, 2017, or 2018. Using a one-step approach, IPD across studies were pooled. Adjusted odds ratios (ORs) and 95% CIs were derived from multivariable logistic regression models of each AF combined with major features except threshold growth (excluded because of infrequent reporting). Liver observation clustering was addressed at the study and participant levels through random intercepts. Risk of bias was assessed using a composite reference standard and Quality Assessment of Diagnostic Accuracy Studies 2. Results Twenty studies comprising 3091 observations (2456 adult participants; mean age, 59 years ± 11 [SD]; 1849 [75.3%] men) were included. In total, 89% (eight of nine) of AFs favoring malignancy were associated with malignancy and/or HCC, 80% (four of five) of AFs favoring HCC were associated with HCC, and 57% (four of seven) of AFs favoring benignity were negatively associated with HCC and/or malignancy. Nonenhancing capsule (OR = 3.50 [95% CI: 1.53, 8.01]) had the strongest association with HCC. Diffusion restriction (OR = 14.45 [95% CI: 9.82, 21.27]) and mild-moderate T2 hyperintensity (OR = 10.18 [95% CI: 7.17, 14.44]) had the strongest association with malignancy. The strongest negative associations with HCC were parallels blood pool enhancement (OR = 0.07 [95% CI: 0.01, 0.49]) and marked T2 hyperintensity (OR = 0.18 [95% CI: 0.07, 0.45]). Seventeen studies (85%) had a high risk of bias. Conclusion Most LI-RADS AFs were independently associated with HCC, malignancy, or benignity as intended when adjusting for major features. © RSNA, 2024 Supplemental material is available for this article. See also the editorial by Crivellaro in this issue.

Cardiac reverse remodeling in primary mitral regurgitation: mitral valve replacement vs. mitral valve repair

BackgroundWhen feasible, guidelines recommend mitral valve repair (MVr) over mitral valve replacement (MVR) to treat primary mitral regurgitation (MR), based upon historic outcome studies and transthoracic echocardiography (TTE) reverse remodeling studies. Cardiovascular magnetic resonance (CMR) offers reference standard biventricular assessment with superior MR quantification compared to TTE. Using serial CMR in primary MR patients, we aimed to investigate cardiac reverse remodeling and residual MR post-MVr vs MVR with chordal preservation. Methods83 patients with ≥ moderate-severe MR on TTE were prospectively recruited. 6-min walk tests (6MWT) and CMR imaging including cine imaging, aortic/pulmonary through-plane phase contrast imaging, T1 maps and late-gadolinium-enhanced (LGE) imaging were performed at baseline and 6 months after mitral surgery or watchful waiting (control group). Results72 patients completed follow-up (Controls = 20, MVr = 30 and MVR = 22). Surgical groups demonstrated comparable baseline cardiac indices and co-morbidities. At 6-months, MVr and MVR groups demonstrated comparable improvements in 6MWT distances (+ 57 ± 54 m vs + 64 ± 76 m respectively, p = 1), reduced indexed left ventricular end-diastolic volumes (LVEDVi; − 29 ± 21 ml/m2 vs − 37 ± 22 ml/m2 respectively, p = 0.584) and left atrial volumes (− 23 ± 30 ml/m2 and − 39 ± 26 ml/m2 respectively, p = 0.545). At 6-months, compared with controls, right ventricular ejection fraction was poorer post-MVr (47 ± 6.1% vs 53 ± 8.0% respectively, p = 0.01) compared to post-MVR (50 ± 5.7% vs 53 ± 8.0% respectively, p = 0.698). MVR resulted in lower residual MR-regurgitant fraction (RF) than MVr (12 ± 8.0% vs 21 ± 11% respectively, p = 0.022). Baseline and follow-up indices of diffuse and focal myocardial fibrosis (Native T1 relaxation times, extra-cellular volume and quantified LGE respectively) were comparable between groups. Stepwise multiple linear regression of indexed variables in the surgical groups demonstrated baseline indexed mitral regurgitant volume as the sole multivariate predictor of left ventricular (LV) end-diastolic reverse remodelling, baseline LVEDVi as the most significant independent multivariate predictor of follow-up LVEDVi, baseline indexed LV end-systolic volume as the sole multivariate predictor of follow-up LV ejection fraction and undergoing MVR (vs MVr) as the most significant (p < 0.001) baseline multivariate predictor of lower residual MR. ConclusionIn primary MR, MVR with chordal preservation may offer comparable cardiac reverse remodeling and functional benefits at 6-months when compared to MVr. Larger, multicenter CMR studies are required, which if the findings are confirmed could impact future surgical practice.

Validation of a screening tool for labor and sex trafficking among emergency department patients.

ObjectivePatients with labor and sex trafficking experiences seek healthcare while and after being trafficked. Their trafficking experiences are often unrecognized by clinicians who lack a validated tool to systematically screen for trafficking. We aimed to derive and validate a brief, comprehensive trafficking screening tool for use in healthcare settings.MethodsPatients were randomly selected to participate in this prospective study based on time of arrival. Data collectors administered 5 dichotomous index questions and a reference standard trafficking assessment tool that requires 30 to 60 minutes to administer. Data collection was from June 2016 to January 2021. Data from patients in 5 New York City (NYC) emergency departments (EDs) were used for tool psychometric derivation, and data from patients in a Fort Worth ED were used for external validation. Clinically stable ED adults (aged ≥18 years) were eligible to participate. Candidate questions were selected from the Trafficking Victim Identification Tool (TVIT). The study outcome measurement was a determination of a participant having a lifetime experience of labor and/or sex trafficking based on the interpretation of the reference standard interview, the TVIT.ResultsOverall, 4127 ED patients were enrolled. In the derivation group, the reference standard identified 36 (1.1%) as positive for a labor and/or sex trafficking experience. In the validation group, 12 (1.4%) were positive by the reference standard. Rapid Appraisal for Trafficking (RAFT) is a new 4‐item trafficking screening tool: in the derivation group, RAFT was 89% sensitive (95% confidence interval [CI], 79%–99%) and 74% specific (95% CI, 73%–76%) and in the external validation group, RAFT was 100% sensitive (95% CI, 100%–100%) and 61% specific (95% CI, 56%–65%).ConclusionsThe rapid, 4‐item RAFT screening tool demonstrated good sensitivity compared with the existing, resource‐intensive reference standard tool. RAFT may enhance the detection of human trafficking in EDs. Additional multicenter studies and research on RAFT's implementation are needed.

Informant Questionnaire on Cognitive Decline in the Elderly (IQCODE) for the early detection of dementia across a variety of healthcare settings.

The Informant Questionnaire for Cognitive Decline in the Elderly (IQCODE) is a structured interview based on informant responses that is used to assess for possible dementia. IQCODE has been used for retrospective or contemporaneous assessment of cognitive decline. There is considerable interest in tests that may identify those at future risk of developing dementia. Assessing a population free of dementia for the prospective development of dementia is an approach often used in studies of dementia biomarkers. In theory, questionnaire-based assessments, such as IQCODE, could be used in a similar way, assessing for dementia that is diagnosed on a later (delayed) assessment. To determine the accuracy of the informant-based questionnaire IQCODE for the early detection of dementia across a variety of health care settings. We searched these sources on 16 January 2016: ALOIS (Cochrane Dementia and Cognitive Improvement Group), MEDLINE Ovid SP, Embase Ovid SP, PsycINFO Ovid SP, BIOSIS Previews on Thomson Reuters Web of Science, Web of Science Core Collection (includes Conference Proceedings Citation Index) on Thomson Reuters Web of Science, CINAHL EBSCOhost, and LILACS BIREME. We also searched sources specific to diagnostic test accuracy: MEDION (Universities of Maastricht and Leuven); DARE (Database of Abstracts of Reviews of Effects, in the Cochrane Library); HTA Database (Health Technology Assessment Database, in the Cochrane Library), and ARIF (Birmingham University). We checked reference lists of included studies and reviews, used searches of included studies in PubMed to track related articles, and contacted research groups conducting work on IQCODE for dementia diagnosis to try to find additional studies. We developed a sensitive search strategy; search terms were designed to cover key concepts using several different approaches run in parallel, and included terms relating to cognitive tests, cognitive screening, and dementia. We used standardised database subject headings, such as MeSH terms (in MEDLINE) and other standardised headings (controlled vocabulary) in other databases, as appropriate. We selected studies that included a population free from dementia at baseline, who were assessed with the IQCODE and subsequently assessed for the development of dementia over time. The implication was that at the time of testing, the individual had a cognitive problem sufficient to result in an abnormal IQCODE score (defined by the study authors), but not yet meeting dementia diagnostic criteria. We screened all titles generated by the electronic database searches, and reviewed abstracts of all potentially relevant studies. Two assessors independently checked the full papers for eligibility and extracted data. We determined quality assessment (risk of bias and applicability) using the QUADAS-2 tool, and reported quality using the STARDdem tool. From 85 papers describing IQCODE, we included three papers, representing data from 626 individuals. Of this total, 22% (N = 135/626) were excluded because of prevalent dementia. There was substantial attrition; 47% (N = 295) of the study population received reference standard assessment at first follow-up (three to six months) and 28% (N = 174) received reference standard assessment at final follow-up (one to three years). Prevalence of dementia ranged from 12% to 26% at first follow-up and 16% to 35% at final follow-up. The three studies were considered to be too heterogenous to combine, so we did not perform meta-analyses to describe summary estimates of interest. Included patients were poststroke (two papers) and hip fracture (one paper). The IQCODE was used at three thresholds of positivity (higher than 3.0, higher than 3.12 and higher than 3.3) to predict those at risk of a future diagnosis of dementia. Using a cut-off of 3.0, IQCODE had a sensitivity of 0.75 (95%CI 0.51 to 0.91) and a specificity of 0.46 (95%CI 0.34 to 0.59) at one year following stroke. Using a cut-off of 3.12, the IQCODE had a sensitivity of 0.80 (95%CI 0.44 to 0.97) and specificity of 0.53 (95C%CI 0.41 to 0.65) for the clinical diagnosis of dementia at six months after hip fracture. Using a cut-off of 3.3, the IQCODE had a sensitivity of 0.84 (95%CI 0.68 to 0.94) and a specificity of 0.87 (95%CI 0.76 to 0.94) for the clinical diagnosis of dementia at one year after stroke. In generaI, the IQCODE was sensitive for identification of those who would develop dementia, but lacked specificity. Methods for both excluding prevalent dementia at baseline and assessing for the development of dementia were varied, and had the potential to introduce bias. Included studies were heterogenous, recruited from specialist settings, and had potential biases. The studies identified did not allow us to make specific recommendations on the use of the IQCODE for the future detection of dementia in clinical practice. The included studies highlighted the challenges of delayed verification dementia research, with issues around prevalent dementia assessment, loss to follow-up over time, and test non-completion potentially limiting the studies. Future research should recognise these issues and have explicit protocols for dealing with them.

Mini-Cog for the detection of dementia within a secondary care setting.

The diagnosis of Alzheimer's disease dementia and other dementias relies on clinical assessment. There is a high prevalence of cognitive disorders, including undiagnosed dementia in secondary care settings. Short cognitive tests can be helpful in identifying those who require further specialist diagnostic assessment; however, there is a lack of consensus around the optimal tools to use in clinical practice. The Mini-Cog is a short cognitive test comprising three-item recall and a clock-drawing test that is used in secondary care settings. The primary objective was to determine the accuracy of the Mini-Cog for detecting dementia in a secondary care setting. The secondary objectives were to investigate the heterogeneity of test accuracy in the included studies and potential sources of heterogeneity. These potential sources of heterogeneity will include the baseline prevalence of dementia in study samples, thresholds used to determine positive test results, the type of dementia (Alzheimer's disease dementia or all causes of dementia), and aspects of study design related to study quality. We searched the following sources in September 2012, with an update to 12 March 2019: Cochrane Dementia Group Register of Diagnostic Test Accuracy Studies, MEDLINE (OvidSP), Embase (OvidSP), BIOSIS Previews (Web of Knowledge), Science Citation Index (ISI Web of Knowledge), PsycINFO (OvidSP), and LILACS (BIREME). We made no exclusions with regard to language of Mini-Cog administration or language of publication, using translation services where necessary. We included cross-sectional studies and excluded case-control designs, due to the risk of bias. We selected those studies that included the Mini-Cog as an index test to diagnose dementia where dementia diagnosis was confirmed with reference standard clinical assessment using standardised dementia diagnostic criteria. We only included studies in secondary care settings (including inpatient and outpatient hospital participants). We screened all titles and abstracts generated by the electronic database searches. Two review authors independently checked full papers for eligibility and extracted data. We determined quality assessment (risk of bias and applicability) using the QUADAS-2 tool. We extracted data into two-by-two tables to allow calculation of accuracy metrics for individual studies, reporting the sensitivity, specificity, and 95% confidence intervals of these measures, summarising them graphically using forest plots. Three studies with a total of 2560 participants fulfilled the inclusion criteria, set in neuropsychology outpatient referrals, outpatients attending a general medicine clinic, and referrals to a memory clinic. Only n = 1415 (55.3%) of participants were included in the analysis to inform evaluation of Mini-Cog test accuracy, due to the selective use of available data by study authors. There were concerns related to high risk of bias with respect to patient selection, and unclear risk of bias and high concerns related to index test conduct and applicability. In all studies, the Mini-Cog was retrospectively derived from historic data sets. No studies included acute general hospital inpatients. The prevalence of dementia ranged from 32.2% to 87.3%. The sensitivities of the Mini-Cog in the individual studies were reported as 0.67 (95% confidence interval (CI) 0.63 to 0.71), 0.60 (95% CI 0.48 to 0.72), and 0.87 (95% CI 0.83 to 0.90). The specificity of the Mini-Cog for each individual study was 0.87 (95% CI 0.81 to 0.92), 0.65 (95% CI 0.57 to 0.73), and 1.00 (95% CI 0.94 to 1.00). We did not perform meta-analysis due to concerns related to risk of bias and heterogeneity. This review identified only a limited number of diagnostic test accuracy studies using Mini-Cog in secondary care settings. Those identified were at high risk of bias related to patient selection and high concerns related to index test conduct and applicability. The evidence was indirect, as all studies evaluated Mini-Cog differently from the review question, where it was anticipated that studies would conduct Mini-Cog and independently but contemporaneously perform a reference standard assessment to diagnose dementia. The pattern of test accuracy varied across the three studies. Future research should evaluate Mini-Cog as a test in itself, rather than derived from other neuropsychological assessments. There is also a need for evaluation of the feasibility of the Mini-Cog for the detection of dementia to help adequately determine its role in the clinical pathway.

Quantification of mitral regurgitation and biventricular assessment is feasible during continuous supine exercise cardiovascular magnetic resonance in primary mitral regurgitation patients

Abstract Funding Acknowledgements Type of funding sources: None. Background Biventricular volume and great vessel flow assessment during continuous supine free-breathing exercise cardiovascular magnetic resonance (Ex-CMR) is feasible in healthy volunteers using Compressed SENSEx3 (CS3) sequences1. Exercise transthoracic echocardiography (TTE) provides prognostic information in primary mitral regurgitation (MR). Resting CMR offers reference standard biventricular assessment and MR quantification with superior reproducibility to TTE. Ex-CMR assessment of biventricular volumes and quantitated MR may offer additional prognostic information. Purpose Determine the feasibility of biventricular assessment and MR quantification in primary MR patients during continuous supine Ex-CMR using the recently validated protocol1. Methods 10 asymptomatic patients with at least moderate primary MR on TTE (8 male, median age 62, 55-67years interquartile range) underwent continuous in-scanner (1.5T Philips Ingenia) supine cycle ergometer (Lode BV) Ex-CMR. Target heart rates (THR) were individually prescribed using heart rate reserve (HRR) and age predicted maximal heart rate model. Participants exercised for 2-minutes at no resistance, then an increase of 25-Watts every 2-minutes until THR achieved at low (30-39% HRR), then moderate (40-59% HRR) stages. CMR imaging: free-breathing CS3 respiratory navigated short axis cine imaging and free-breathing CS3 aortic phase-contrast magnetic-resonance at rest, low and moderate exercise stages. MR was quantified indirectly from left ventricular (LV) and aortic stroke volumes. Intra/inter-observer reproducibility was assessed by coefficient of variance (CV). Results All patients completed the Ex-CMR protocol without complication. During exercise, no statistically significant changes occurred in LV volumes, global left ventricular ejection fraction (LVEF), right ventricular end-diastolic or systolic volumes (Table 1). From rest to low and moderate exercise: right ventricular ejection fraction increased (55 ± 5.4% to 60 ± 6.0% and 63 ± 6.6% respectively, p = 0.001) and MR fraction decreased (40 ± 14% to 36 ± 11% and 30 ± 15% respectively, p = 0.006) allowing effective forward LVEF (aortic stroke volume/ left ventricular end-diastolic volume) to increase (38 ± 9.3% to 43 ± 9.3% and 46 ± 11% respectively, p = 0.004). Intra-observer reproducibility (Table 2) was excellent (CV &amp;lt;10%), except right ventricular stroke volumes (RVSV) during low, MR volumes and fraction during moderate and right ventricular end-systolic volumes (RVESV) during both exercise stages, which were good (CV10-20%). Inter-observer reproducibility (Table 2) was excellent (CV &amp;lt; 10%), except RVESV and MR volumes at all stages, LV end-systolic volumes during low and MR fraction during moderate exercise, which were good (CV 10-20%). Conclusion Biventricular assessment and MR quantification during continuous supine Ex-CMR is feasible in asymptomatic primary MR patients, further research assessing the techniques prognostic ability is now warranted.

Validation of the 4AT tool for delirium assessment in specialist palliative care settings: protocol of a prospective diagnostic test accuracy study [version 1; peer review: 2 approved

BackgroundDelirium is a serious and distressing neuropsychiatric condition, which is prevalent across all palliative care settings. Hypoactive delirium is particularly common, but difficult to recognize, partly due to overlapping symptoms with depression and dementia. Delirium screening tools can lead to earlier identification and hence better management of patients. The 4AT (4 ‘A’s Test) is a brief tool for delirium detection, designed for use in clinical practice. It has been validated in 17 studies in over 3,700 patients. The test is currently used in specialist palliative care units, but has not been validated in this setting. The aim of the study is to determine the diagnostic accuracy of the 4AT for delirium detection against a reference standard, in hospice inpatients.Methods240 participants will be recruited from the inpatient units of two hospices in Scotland. If a patient lacks capacity to consent, agreement will be sought from a legal proxy. Each participant will complete the 4AT and a reference standard assessment based on the diagnostic delirium criteria in the fifth edition of the Diagnostic and Statistical Manual of Mental Disorders (DSM-5). This will be supplemented by tests of cognition and attention, including reverse days of the week, counting down from 20 to 1, Vigilance 'A', the Observational Scale for Level of Arousal, the modified Richmond Agitation Sedation Scale and the Delirium Rating Scale-Revised-98. The assessments will be conducted in a randomized order by two independent clinicians, who will be blinded to the results until both are complete. Primary outcomes will be the sensitivity and specificity of the 4AT in detecting delirium.DiscussionThe findings will inform clinical practice regarding delirium assessment in palliative care settings.Trial registrationISRCTN ISRCTN97417474 (21/02/2020).

The 4-DSD: A New Tool to Assess Delirium Superimposed on Moderate to Severe Dementia

ObjectivesThe purpose of this study was to create, standardize, and validate a new instrument, named 4-DSD, and determine its diagnostic accuracy in the diagnosis of delirium in subjects with moderate to severe dementia. DesignMulticenter cross-sectional observational study. Setting and ParticipantsOlder patients consecutively admitted to acute and rehabilitation hospital wards. MeasuresThe DSM-5 was used as the reference standard delirium assessment. The presence and severity of dementia was defined using the AD8 and the Global Deterioration Scale (GDS). The 4-DSD is a 4-item tool that ranges from 0 to 12. Item 1 measures alertness, item 2 altered function, item 3 attention, and item 4 acute change or fluctuation in mental status. ResultsA total of 134 patients were included in the study. Most of the patients were enrolled in acute hospital wards (60%), with 40% in rehabilitation settings. A minority of the patients were categorized with moderate dementia, with a GDS score of 5 (4%). Most of the patients were in the moderate-severe stage with a GDS score ≤6 (77%); 19% were classed as severe, with a GDS score of 7. A 4-DSD cutoff score ≥5 had a sensitivity of 80% and specificity of 80% with a positive predictive value (PPV) of 67% and a negative predictive value (NPV) of 89%. In the subgroup with moderate-severe dementia (n = 108), the sensitivity and the specificity were 79% and 82%, respectively, with a PPV and NPV of 62% and 92%. In the subgroup with severe dementia (n = 26) the sensitivity was 82% and the specificity 56% with a PPV of 78% and a NPV of 63%. Conclusions and ImplicationsThe availability of a specific tool to detect delirium in patients with moderate-severe dementia has important clinical and research implications, allowing all health care providers to improve their ability to identify it.

Screening for delirium in the intensive care unit using eDIS-ICU – A purpose-designed app: A pilot study

IntroductionDelirium, a common complication of an intensive care unit (ICU) admission, is inconsistently diagnosed by clinicians. Current screening tools require specialist expertise and/or training. Some are time-consuming to administer, and reliability in routine clinical practice is questionable. An innovative app designed to enable efficient and sensitive screening for delirium without specialist training (eDIS-ICU) has recently been described. This pilot study compared the eDIS-ICU against the reference standard expert assessment using DSM-V (Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition) criteria and the Confusion Assessment Method for the ICU (CAM-ICU). MethodsIn this prospective, single-centre pilot study, a convenience sample of 29 ICU patients were recruited at a tertiary referral hospital between November 2018 and August 2019. After obtaining written consent, demographic and clinical data were collected, and the patients were screened for delirium using eDIS-ICU and CAM-ICU by two clinician researchers in random order. The patients were also assessed for presence of delirium independently by an expert clinician using a structured interview to diagnose as per DSM-V criteria. The results of screening and diagnosis were tabulated to allow comparison of screening tools against diagnosis; sensitivity and specificity of the tools were calculated. ResultsSeven participants were diagnosed with delirium as per DSM-V criteria. The eDIS-ICU tool correctly identified six of these participants compared with two identified by CAM-ICU. The sensitivity of the eDIS-ICU tool was 86% (95% confidence interval [CI] = 81.5–100.0) compared with 29% (95% CI = 5.1–69.7) for CAM-ICU (p < 0.05), and the specificity was 73% (95% CI = 81.5–100.0) versus 96% (95% CI = 75.1–99.8), respectively. ConclusionThe simple and novel eDIS-ICU delirium screening tool was noninferior to the CAM-ICU in detecting delirium as per DSM-V criteria. A definitive validation study is warranted.

MHealth developmental screening for preschool children in low-income communities.

Children are often only identified with a developmental delay when they enter primary school due to developmental delays hindering academic progress. Detection of at-risk children in low-income communities is typically unavailable due to several challenges. This study validated an mHealth-based developmental screening tool as a potential time- and cost-effective way of delivering services for preschool children. This cross-sectional within-subject study screened 276 preschool children from low-income communities using the mHealth Parents' Evaluation of Developmental Status (PEDS) tools. The mHealth PEDS tools' performance was then evaluated by comparing caregiver concerns according to the PEDS tools with results obtained using a reference standard assessment tool, the Vineland-3. The mHealth PEDS tools identified 237 (85.9%) of children at risk of developmental delay compared to 80.1% (n = 221) of children identified with a developmental delay using the Vineland-3. Sensitivity of the PEDS tools was high (92.6%) with low specificity (22.5%) using US standardised criteria. Literacy skills were found to be most delayed, according to the PEDS: DM (89.3%; n = 142) and Vineland-3 (87.1%; n = 134). Low specificity of the prescribed criteria may require the implementation of adapted referral criteria within low socio-economic status (SES) settings. The mHealth PEDS tools may still be valuable for preschool developmental surveillance of children within low SES settings. It is recommended, however, that children who are identified with a developmental concern undergo a second screen to reduce false positives.

Montreal cognitive assessment for evaluating cognitive impairment in Huntington’s disease: a systematic review

This study aims to systematically review evidence of the accuracy of the Montreal Cognitive Assessment (MoCA) for evaluating the presence of cognitive impairment in patients with Huntington's disease (HD) and to outline the quality and quantity of research evidence available about the use of the MoCA in this population. We conducted a systematic literature review, searching four databases from inception until April 2020. We identified 26 studies that met the inclusion criteria: two case-control studies comparing the MoCA to a battery of tests, three studies comparing MoCA to Mini-Mental State Examination, two studies estimating the prevalence of cognitive impairment in individuals with HD and 19 studies or clinical trials in which the MoCA was used as an instrument for the cognitive assessment of participants with HD. We found no cross-sectional studies in which participants received the index test (MoCA) and a reference standard diagnostic assessment composed of an extensive neuropsychological battery. The publication period ranged from 2010 to 2020. In patients with HD, the MoCA provides information about disturbances in general cognitive function. Even if the MoCA demonstrated good sensitivity and specificity when used at the recommended threshold score of 26, further cross-sectional studies are required to examine the optimum cutoff score for detecting cognitive impairments in patients with HD. Moreover, more studies are necessary to determine whether the MoCA adequately assesses cognitive status in individuals with HD.

Parkinson's Disease-Cognitive Rating Scale for Evaluating Cognitive Impairment in Parkinson's Disease: A Systematic Review.

The aim of the present systematic review was to examine the evidence on the accuracy and psychometric properties of the Parkinson’s Disease-Cognitive Rating Scale (PD-CRS) for evaluating the presence of cognitive impairment in patients with Parkinson’s disease (PD) as well as to highlight the quality and quantity of research available on the use of the PD-CRS in this population. We searched four databases from inception until July 2020. Eight studies, published between 2008 and 2020, met the inclusion criteria: One cross-sectional study in which participants were assessed with the index test (PD-CRS) and a reference standard diagnostic assessment, in accordance with the Level II criteria of the International Parkinson and Movement Disorder Society (MDS); one case-control study comparing the PD-CRS to an extensive battery of tests (i.e., MDS Level II diagnosis); and six studies comparing the PD-CRS to other short cognitive batteries. In patients with Parkinson’s disease, the PD-CRS test provides information about cortical and sub-cortical cognitive functions. Even if it demonstrated good psychometric properties, the results regarding the optimal threshold for detecting mild cognitive impairment and dementia in PD are somewhat inconsistent. Further cross-sectional studies are necessary to examine the optimum cut-off score for detecting cognitive dysfunction in PD patients.

Ultrabrief Screens for Detecting Delirium in Postoperative Cognitively Intact Older Adults.

The authors' sought to develop an ultrabrief screen for postoperative delirium in cognitively intact patients older than 70 years undergoing major elective surgery. All possible combinations of one-, two- and three-item screens and their sensitivities, specificities, and 95% confidence intervals were calculated and compared with the delirium reference standard Confusion Assessment Method (CAM). Among the 560 participants (mean age, 77 years; 58% women), delirium occurred in 134 (24%). We considered 1,100 delirium assessments from postoperative days 1 and 2. The screen with the best overall performance consisted of three items: (1) Patient reports feeling confused, (2) Months of the year backward, and (3) "Does the patient appear sleepy?" with sensitivity of 92% and specificity of 72%. This brief, three-item screen rules out delirium quickly, identifies a subset of patients who require further testing, and may be an important tool to improve recognition of postoperative delirium.

Validation of an algorithm based on administrative data to detect new onset of atrial fibrillation after cardiac surgery

IntroductionPostoperative atrial fibrillation (POAF) is a frequent complication of cardiac surgery associated with important morbidity, mortality, and costs. To assess the effectiveness of preventive interventions, an important prerequisite is to have access to accurate measures of POAF incidence. The aim of this study was to develop and validate such a measure.MethodsA validation study was conducted at two large Canadian university health centers. First, a random sample of 976 (10.4%) patients who had cardiac surgery at these sites between 2010 and 2016 was generated. Then, a reference standard assessment of their medical records was performed to determine their true POAF status on discharge (positive/negative). The accuracy of various algorithms combining diagnostic and procedure codes from: 1) the current hospitalization, and 2) hospitalizations up to 6 years before the current hospitalization was assessed in comparison with the reference standard. Overall and site-specific estimates of sensitivity, specificity, positive (PPV), and negative (NPV) predictive values were generated, along with their 95%CIs.ResultsUpon manual review, 324 (33.2%) patients were POAF-positive. Our best-performing algorithm combining data from both sites used a look-back window of 6 years to exclude patients previously known for AF. This algorithm achieved 70.4% sensitivity (95%CI: 65.1–75.3), 86.0% specificity (95%CI: 83.1–88.6), 71.5% PPV (95%CI: 66.2–76.4), and 85.4% NPV (95%CI: 82.5–88.0). However, significant site-specific differences in sensitivity and NPV were observed.ConclusionAn algorithm based on administrative data can identify POAF patients with moderate accuracy. However, site-specific variations in coding practices have significant impact on accuracy.

Preliminary Assessment of an Optical Flow Method (OFM) for Nonrigid Registration and Temporal Subtraction (TS) of Serial CT Examinations to Facilitate Evaluation of Interval Change in Metastatic Lung Nodules

Rationale and ObjectivesAccurate assessment of size change of lung nodules on chest computed tomography (CT) is important for diagnosis and response assessment. However, manual methods are time-consuming and error-prone. We therefore assessed whether an optical flow method (OFM) with temporal subtraction (TS) can facilitate detection and quantification of lung nodule change on serial CT datasets. Materials and MethodsSerial chest CT examinations were selected from 12 patients with multiple pulmonary metastases. Lung nodules were evaluated for change in size using: (1) OFM with TS and (2) reference standard visual and manual assessment. Average time required to assess interval change using both methods was recorded and compared. Concordance of agreement between OFM with TS and reference standard assessment for nodule change was examined. Results285 solid pulmonary nodules were evaluated. The average time per nodule to assess interval change in nodule size by OFM with TS (mean 1.15 + 0.5 minutes) was significantly less (P = 0.02) than that the reference standard approach (mean 1.56 + 0.5 minutes). Agreement between OFM with TS and reference standard occurred for 63.2% of nodules overall (kappa = 0.50, standard error 0.35, P< 0.00001), and significantly increased with larger nodule size (kappa = 0.48 for nodules <5 mm; kappa = 0.94 for nodules >20 mm, P < 0.0001). ConclusionsThis preliminary study demonstrates the feasibility of an OFM with TS to assess for interval change in metastatic lung nodules on serial CT examinations with significantly improved reading speed and moderate agreement relative to reference standard assessment. Agreement improved with larger nodule size.

A smartphone-based test for the assessment of attention deficits in delirium: A case-control diagnostic test accuracy study in older hospitalised patients.

BackgroundDelirium is a common and serious acute neuropsychiatric syndrome which is often missed in routine clinical care. Inattention is the core cognitive feature. Diagnostic test accuracy (including cut-points) of a smartphone Delirium App (DelApp) for assessing attention deficits was assessed in older hospital inpatients.MethodsThis was a case-control study of hospitalised patients aged ≥65 years with delirium (with or without pre-existing cognitive impairment), who were compared to patients with dementia without delirium, and patients without cognitive impairment. Reference standard delirium assessment, which included a neuropsychological test battery, was based on Diagnostic and Statistical Manual of Mental Disorders-5 criteria. A separate blinded assessor administered the DelApp arousal assessment (score 0–4) and attention task (0–6) yielding an overall score of 0 to 10 (lower scores indicate poorer performance). Analyses included receiver operating characteristic curves and sensitivity and specificity. Optimal cut-points for delirium detection were determined using Youden’s index.ResultsA total of 187 patients were recruited, mean age 83.8 (range 67–98) years, 152 (81%) women; n = 61 with delirium; n = 61 with dementia without delirium; and n = 65 without cognitive impairment. Patients with delirium performed poorly on the DelApp (median score = 4/10; inter-quartile range 3.0, 5.5) compared to patients with dementia (9.0; 5.5, 10.0) and those without cognitive impairment (10.0; 10.0, 10.0). Area under the curve for detecting delirium was 0.89 (95% Confidence Interval 0.84, 0.94). At an optimal cut-point of ≤8, sensitivity was 91.7% (84.7%, 98.7%) and specificity 74.2% (66.5%, 81.9%) for discriminating delirium from the other groups. Specificity was 68.3% (56.6%, 80.1%) for discriminating delirium from dementia (cut-point ≤6).ConclusionPatients with delirium (with or without pre-existing cognitive impairment) perform poorly on the DelApp compared to patients with dementia and those without cognitive impairment. A cut-point of ≤8/10 is suggested as having optimal sensitivity and specificity. The DelApp is a promising tool for assessment of attention deficits associated with delirium in older hospitalised adults, many of whom have prior cognitive impairment, and should be further validated in representative patient cohorts.

Mini-Cog for the diagnosis of Alzheimer's disease dementia and other dementias within a secondary care setting.

The diagnosis of Alzheimer's disease dementia and other dementias relies on clinical assessment. There is a high prevalence of cognitive disorders, including undiagnosed dementia in secondary care settings. Short cognitive tests can be helpful in identifying those who require further specialist diagnostic assessment; however, there is a lack of consensus around the optimal tools to use in clinical practice. The Mini-Cog is a short cognitive test comprising three-item recall and a clock-drawing test that is used in secondary care settings. The primary objective was to determine the diagnostic accuracy of the Mini-Cog for detecting Alzheimer's disease dementia and other dementias in a secondary care setting. The secondary objectives were to investigate the heterogeneity of test accuracy in the included studies and potential sources of heterogeneity. These potential sources of heterogeneity will include the baseline prevalence of dementia in study samples, thresholds used to determine positive test results, the type of dementia (Alzheimer's disease dementia or all causes of dementia), and aspects of study design related to study quality. We searched the following sources in September 2012, with an update to 12 March 2019: Cochrane Dementia Group Register of Diagnostic Test Accuracy Studies, MEDLINE (OvidSP), Embase (OvidSP), BIOSIS Previews (Web of Knowledge), Science Citation Index (ISI Web of Knowledge), PsycINFO (OvidSP), and LILACS (BIREME). We made no exclusions with regard to language of Mini-Cog administration or language of publication, using translation services where necessary. We included cross-sectional studies and excluded case-control designs, due to the risk of bias. We selected those studies that included the Mini-Cog as an index test to diagnose dementia where dementia diagnosis was confirmed with reference standard clinical assessment using standardised dementia diagnostic criteria. We only included studies in secondary care settings (including inpatient and outpatient hospital participants). We screened all titles and abstracts generated by the electronic database searches. Two review authors independently checked full papers for eligibility and extracted data. We determined quality assessment (risk of bias and applicability) using the QUADAS-2 tool. We extracted data into two-by-two tables to allow calculation of accuracy metrics for individual studies, reporting the sensitivity, specificity, and 95% confidence intervals of these measures, summarising them graphically using forest plots. Three studies with a total of 2560 participants fulfilled the inclusion criteria, set in neuropsychology outpatient referrals, outpatients attending a general medicine clinic, and referrals to a memory clinic. Only n = 1415 (55.3%) of participants were included in the analysis to inform evaluation of Mini-Cog test accuracy, due to the selective use of available data by study authors. There were concerns related to high risk of bias with respect to patient selection, and unclear risk of bias and high concerns related to index test conduct and applicability. In all studies, the Mini-Cog was retrospectively derived from historic data sets. No studies included acute general hospital inpatients. The prevalence of dementia ranged from 32.2% to 87.3%. The sensitivities of the Mini-Cog in the individual studies were reported as 0.67 (95% confidence interval (CI) 0.63 to 0.71), 0.60 (95% CI 0.48 to 0.72), and 0.87 (95% CI 0.83 to 0.90). The specificity of the Mini-Cog for each individual study was 0.87 (95% CI 0.81 to 0.92), 0.65 (95% CI 0.57 to 0.73), and 1.00 (95% CI 0.94 to 1.00). We did not perform meta-analysis due to concerns related to risk of bias and heterogeneity. This review identified only a limited number of diagnostic test accuracy studies using Mini-Cog in secondary care settings. Those identified were at high risk of bias related to patient selection and high concerns related to index test conduct and applicability. The evidence was indirect, as all studies evaluated Mini-Cog differently from the review question, where it was anticipated that studies would conduct Mini-Cog and independently but contemporaneously perform a reference standard assessment to diagnose dementia. The pattern of test accuracy varied across the three studies. Future research should evaluate Mini-Cog as a test in itself, rather than derived from other neuropsychological assessments. There is also a need for evaluation of the feasibility of the Mini-Cog for the diagnosis of dementia to help adequately determine its role in the clinical pathway.

Delirium detection in older acute medical inpatients: a multicentre prospective comparative diagnostic test accuracy study of the 4AT and the confusion assessment method

BackgroundDelirium affects > 15% of hospitalised patients but is grossly underdetected, contributing to poor care. The 4 ‘A’s Test (4AT, www.the4AT.com) is a short delirium assessment tool designed for routine use without special training. The primary objective was to assess the accuracy of the 4AT for delirium detection. The secondary objective was to compare the 4AT with another commonly used delirium assessment tool, the Confusion Assessment Method (CAM).MethodsThis was a prospective diagnostic test accuracy study set in emergency departments or acute medical wards involving acute medical patients aged ≥ 70. All those without acutely life-threatening illness or coma were eligible. Patients underwent (1) reference standard delirium assessment based on DSM-IV criteria and (2) were randomised to either the index test (4AT, scores 0–12; prespecified score of > 3 considered positive) or the comparator (CAM; scored positive or negative), in a random order, using computer-generated pseudo-random numbers, stratified by study site, with block allocation. Reference standard and 4AT or CAM assessments were performed by pairs of independent raters blinded to the results of the other assessment.ResultsEight hundred forty-three individuals were randomised: 21 withdrew, 3 lost contact, 32 indeterminate diagnosis, 2 missing outcome, and 785 were included in the analysis. Mean age was 81.4 (SD 6.4) years. 12.1% (95/785) had delirium by reference standard assessment, 14.3% (56/392) by 4AT, and 4.7% (18/384) by CAM. The 4AT had an area under the receiver operating characteristic curve of 0.90 (95% CI 0.84–0.96). The 4AT had a sensitivity of 76% (95% CI 61–87%) and a specificity of 94% (95% CI 92–97%). The CAM had a sensitivity of 40% (95% CI 26–57%) and a specificity of 100% (95% CI 98–100%).ConclusionsThe 4AT is a short, pragmatic tool which can help improving detection rates of delirium in routine clinical care.Trial registrationInternational standard randomised controlled trial number (ISRCTN) 53388093. Date applied 30/05/2014; date assigned 02/06/2014.

Ultra-brief Screeners for Detecting Delirium Superimposed on Dementia

ObjectiveDelirium superimposed on dementia (DSD) is common, morbid, and costly, yet frequently undiagnosed. Our study aimed to develop a brief screening test to improve health care worker recognition of DSD. Design/Setting/ParticipantsOlder hospitalized adults with dementia were prospectively enrolled from medical and surgical inpatient units of 3 hospitals (2 in Pennsylvania, 1 in Tennessee). MeasuresThe reference standard delirium assessment used Confusion Assessment Method (CAM) criteria and was based on a structured interview including the Mini-Mental State Examination, interviewer observations, and medical record review. To develop the screening test, 1-, 2-, and 3-item combinations from the reference standard assessment were analyzed to determine their sensitivity and specificity in diagnosing delirium presence in a dementia population compared to the reference standard. For multiple-item screeners, error on 1 or more items was considered a positive screen. ResultsOverall, 391 older adults with dementia were enrolled (mean age: 83.9 years, 71.1% female), and 95 (24.4%) developed DSD during their hospitalization, based on the reference standard. The best single-item screen for DSD was “What day of the week is it?” with 84% sensitivity [95% confidence interval (CI): 0.75, 0.91] and 41% specificity (CI: 0.35, 0.47). The best 2-item screen was “list the days of the week backwards” and “What day of the week is it?” with 93% sensitivity (CI: 0.85, 0.97) and 30% specificity (CI: 0.25, 0.36). The best 3-item screen was “list the days of the week backwards,” “What type of place is this? [hospital]” and “Does the patient appear sleepy?” with 94% sensitivity (CI: 0.87, 0.98) and 42% specificity (CI: 0.36, 0.48). Conclusions/ImplicationsWe identified a 3-item DSD screener with excellent sensitivity but limited specificity. This screener can be used to quickly rule out DSD in populations with a high prevalence of dementia and is a promising step toward developing efficient tools for DSD recognition among care providers.