Optimizing the use of expert panel reference diagnoses in diagnostic studies of multidimensional syndromes

Albert F.g. Leentjens ,Patrick M. Bossuyt ,Manuela A. Joore ,Claire Wolfs ,Johan L. Severens ,Ron Handels ,Frans R.j. Verhey ,Pauline Aalten

doi:10.1186/preaccept-1702688938102976

Abstract

BackgroundIn the absence of a gold standard, a panel of experts can be invited to assign a reference diagnosis for use in research. Available literature offers limited guidance on assembling and working with an expert panel for this purpose. We aimed to develop a protocol for an expert panel consensus diagnosis and evaluated its applicability in a pilot project.MethodsAn adjusted Delphi method was used, which started with the assessment of clinical vignettes by 3 experts individually, followed by a consensus discussion meeting to solve diagnostic discrepancies. A panel facilitator ensured that all experts were able to express their views, and encouraged the use of argumentation to arrive at a specific diagnosis, until consensus was reached by all experts. Eleven vignettes of patients suspected of having a primary neurodegenerative disease were presented to the experts. Clinical information was provided stepwise and included medical history, neurological, physical and cognitive function, brain MRI scan, and follow-up assessments over 2 years. After the consensus discussion meeting, the procedure was evaluated by the experts.ResultsThe average degree of consensus for the reference diagnosis increased from 52% after individual assessment of the vignettes to 94% after the consensus discussion meeting. Average confidence in the diagnosis after individual assessment was 85%. This did not increase after the consensus discussion meeting. The process evaluation led to several recommendations for improvement of the protocol.ConclusionA protocol for attaining a reference diagnosis based on expert panel consensus was shown feasible in research practice.

Highlights

In the absence of a gold standard, a panel of experts can be invited to assign a reference diagnosis for use in research
We have developed a protocol for a consensus panel reference diagnosis in Alzheimer's disease (AD) based on clinically relevant decline as judged by a clinician
If for example a patient’s biomarker profile is available to a panel of experts who assign the reference diagnosis it could lead to overestimation of the biomarker’s accuracy, because the association between biomarker and reference standard is artificially inflated. This could occur when the standard medical practice diagnosis of AD-type dementia at follow up is used as a longitudinal reference standard, because biomarkers are often used in medical research practice

Summary

Introduction

In the absence of a gold standard, a panel of experts can be invited to assign a reference diagnosis for use in research. An example for a disease for which no test fulfils the criteria for a gold standard is Alzheimer's disease (AD) It is defined by a gradual onset of symptoms, deterioration of cognition leading to dementia, and no evidence of another medical comorbidity or medication that could affect cognition [4]. A post-mortem neuropathological examination has been criticized for imperfect inter-observer reliability and imperfect association with cognitive impairment or dementia [8,9] Another reference standard is to follow up a patient in the pre-dementia phase until a clinical diagnosis of AD-type dementia can be made. This requires a long follow-up period to ensure that all patients with a neurodegenerative disease at baseline decline to the level of dementia within that period [10,11]

Objectives

Methods

Results

Discussion

Conclusion